The idea
A retrieval system does not usually pass an entire document to a language model. It selects smaller pieces. The boundary of each piece affects whether a retrieved passage still contains the definitions, qualifications and context needed to answer a question.
Design choice
Use the document structure as a starting point: headings, paragraphs and tables carry meaning. A fixed character limit is a useful constraint, not a complete strategy. Preserve the document identity and section heading alongside every piece.
Worked example
A policy exception sits directly below the general rule. Keep both in the same retrievable passage, or include an explicit link that allows the reader to fetch the exception.
Before you ship
- Inspect boundaries in real documents
- Keep table headers with their values
- Measure retrieval with representative questions
Examples are illustrative. Adapt the checks to your own environment.