Data engineering

Connect an output to the work that produced it

Editorial edition · 16 November 2026

The idea

Lineage explains how a dataset or result depends on sources and transformations. It helps answer which reports may be affected by a changed field or failed import. Useful lineage is specific enough to support an action rather than merely drawing a large diagram.

Design choice

Start with dataset-level dependencies and add finer detail where it changes decisions. Record actual run identifiers alongside the intended pipeline design. A documented dependency does not prove that the latest run completed successfully.

Worked example

A renamed source field is traced to two transformations and one dashboard. The owners can test those outputs before the change reaches routine reporting.

Before you ship

  • Track intended and actual dependencies
  • Connect lineage to owners
  • Test impact queries during a change

Examples are illustrative. Adapt the checks to your own environment.

Further reading

W3C provenance overview

Search the library