An evaluation set turns a vague impression of quality into repeatable evidence. Include normal work, difficult edge cases and questions that should not receive a confident answer. A collection of only easy examples can hide the failures that matter most.
EvaluationEvaluation
Retrieval and evaluation before another layer of automation.
Precision and recall describe different aspects of a selection. Precision asks how many selected items are relevant; recall asks how many relevant items were selected. Neither number explains the cost of a particular mistake without context.
EvaluationFor some tasks, a useful refusal is better than a fabricated answer. Abstention gives the system an explicit outcome when available evidence cannot support a response. It should guide the user toward clarification or another channel rather than ending with an opaque failure.
EvaluationA strong average can conceal a weak result for a particular language, document type or request category. Slice analysis groups outcomes by meaningful conditions. It helps teams discover where a system needs a narrower scope or better supporting data.
EvaluationA new prompt, retriever or model can improve one task while making another worse. A change gate compares the proposed version against a known reference. It should consider quality, latency and operational cost rather than one attractive demonstration.
Evaluation