Applied AI

Split documents at meaningful boundaries

Editorial edition · 16 November 2026

The idea

A retrieval system does not usually pass an entire document to a language model. It selects smaller pieces. The boundary of each piece affects whether a retrieved passage still contains the definitions, qualifications and context needed to answer a question.

Design choice

Use the document structure as a starting point: headings, paragraphs and tables carry meaning. A fixed character limit is a useful constraint, not a complete strategy. Preserve the document identity and section heading alongside every piece.

Worked example

A policy exception sits directly below the general rule. Keep both in the same retrievable passage, or include an explicit link that allows the reader to fetch the exception.

Before you ship

  • Inspect boundaries in real documents
  • Keep table headers with their values
  • Measure retrieval with representative questions

Examples are illustrative. Adapt the checks to your own environment.

Further reading

Retrieval-Augmented Generation, Lewis et al.

Search the library