RAG evaluation should separate retrieval failure from generation failure. If the right passage never reaches the model, prompt changes can improve tone without fixing the answer. Start with a small evaluation set that identifies the expected source for each query.
During testing, record which document chunks were retrieved and whether their metadata matched the intended access rules. A correct answer from an unauthorized document is still a system failure. Tune chunking or ranking only after the failed cases are classified, then evaluate how the model responds when retrieval returns weak or conflicting evidence.