The idea
For some tasks, a useful refusal is better than a fabricated answer. Abstention gives the system an explicit outcome when available evidence cannot support a response. It should guide the user toward clarification or another channel rather than ending with an opaque failure.
Design choice
Define conditions for insufficient evidence and evaluate them on difficult questions. Do not treat a language model's verbal confidence as a calibrated probability. Measure both unsupported answers and unnecessary refusals.
Worked example
A question references a document absent from the collection. The assistant identifies the missing source and asks for it instead of inventing a summary.
Before you ship
- Define a helpful fallback response
- Count unnecessary refusals
- Test absent and contradictory evidence
Examples are illustrative. Adapt the checks to your own environment.