In OpenTelemetry, each span, log, and metric comes with a timestamp baked in. These are generated on the systems emitting the telemetry and are sent to Honeycomb. Honeycomb trusts that the timestamps are correct, even if they're older, so that backfilling or un-sticking a pipeline will fill in the data properly. This also allows batching to create dozens of spans that are microseconds long while still arriving in the same batch.
Sometimes a situation arises where you want to adjust these timestamps.
Example
Maybe one system doesn't have proper daylight savings time and GMT ends up being wrong for some of the year. To fix a situation like this, OpenTelemetry Transform Language can be used by the Transform Processor to adjust the timestamps.
processors:
transform/timeshift:
trace_statements:
- context: span
statements:
- set(start_time, start_time + Duration("1h"))
- set(end_time, end_time + Duration("1h"))
- set(attributes["ottl.added_time"], "1 hour")The first two of the three statements in this transform processor definition add 1 hour to the start and end timestamps.
Telemetry Ergonomics
The third statement adds an attribute to let the viewers at home know that the span was modified. If there is reason to suspect the timestamp is wrong or the way GMT is handled was fixed, your users can see that there was a modification and ask that it be changed.
Future Spans
If you add too much time and the timestamps end up in the future, Honeycomb will set them back to ingest time. Honeycomb does not support sending future-dated spans.