LoRA, retrieval, and evaluation · TU Dresden project
Cultural Question Answering
I built a cultural QA system combining Llama 3 8B, LoRA fine-tuning, FAISS retrieval, constrained outputs, and benchmark-driven error analysis.
01
The challenge
Answer country-specific cultural questions while keeping both multiple-choice and short-answer outputs measurable and reproducible.
02
The approach
Fine-tuned with LoRA, retrieved country-specific Wikipedia context through FAISS and SentenceTransformers, and added deterministic inference and output constraints for evaluation.
Retrieval and adaptation
The project tests fine-tuning and country-specific retrieval as complementary ways to improve answers.
Outputs designed for evaluation
Formatting constraints, post-processing, and error analysis reduce ambiguity between a model response and a measurable result.
03
The outcome
Reported combined accuracy improved from 0.47 to 0.78 on the project benchmark.