Analytics

Choose an observation window before the result

Editorial edition · 16 November 2026

The idea

An experiment needs enough time to observe the behaviour it intends to change. A short window may miss delayed effects or ordinary weekly variation. Repeatedly changing the end date after looking at results makes interpretation harder.

Design choice

Specify the outcome, population and observation period in advance. Account for delayed completion and incomplete records. If monitoring requires early stopping, use a planned method rather than an improvised reaction to a favourable number.

Worked example

A delivery-process test follows each shipment until its normal completion window has passed. Orders still in transit are not silently counted as successful or excluded without explanation.

Before you ship

  • Define the observation period upfront
  • Handle incomplete outcomes explicitly
  • Document stopping rules

Examples are illustrative. Adapt the checks to your own environment.

Further reading

OpenTelemetry metrics

Search the library