Applied AI

Allow a system to say it lacks evidence

Editorial edition · 16 November 2026

The idea

For some tasks, a useful refusal is better than a fabricated answer. Abstention gives the system an explicit outcome when available evidence cannot support a response. It should guide the user toward clarification or another channel rather than ending with an opaque failure.

Design choice

Define conditions for insufficient evidence and evaluate them on difficult questions. Do not treat a language model's verbal confidence as a calibrated probability. Measure both unsupported answers and unnecessary refusals.

Worked example

A question references a document absent from the collection. The assistant identifies the missing source and asks for it instead of inventing a summary.

Before you ship

  • Define a helpful fallback response
  • Count unnecessary refusals
  • Test absent and contradictory evidence

Examples are illustrative. Adapt the checks to your own environment.

Further reading

Model evaluation, scikit-learn

Search the library