The dashboard was real time and the pipeline was hourly

# Self-authored for the EML case corpus (no external origin). The dashboard
# refreshes every five seconds over a push connection and shows when it last
# updated. What that timestamp is the age of is computed below.
#
# The page is genuinely live. It is not a screenshot and not a five-minute
# poll: a push connection delivers the new value, the query behind it returns
# in tens of milliseconds, and the page carries a visible last-updated
# timestamp rather than leaving the reader to guess. Somebody built that
# deliberately, and every part of it is true.
#
# The timestamp says when the PAGE refreshed. The table it reads is written by
# a batch that runs once an hour and takes fourteen minutes, so the freshest
# number the page can show was computed some time ago and the page has no way
# to say so.
#
# Four of the six alerts wired to this table have windows shorter than an hour.

5 => refresh_seconds
40 => query_to_pixel_ms
60 => pipeline_interval_minutes
14 => pipeline_runtime_minutes
60 => seconds_in_a_minute
1 => timestamps_shown_on_the_page
0 => timestamps_showing_when_the_data_was_produced
6 => alerts_wired_to_this_table
4 => alerts_whose_window_is_shorter_than_the_interval

pipeline_interval_minutes + pipeline_runtime_minutes => worst_case_data_age_minutes
worst_case_data_age_minutes * seconds_in_a_minute => worst_case_data_age_seconds
int(refresh_seconds * 10000 / worst_case_data_age_seconds) => displayed_age_per_myriad_of_actual
int(alerts_whose_window_is_shorter_than_the_interval * 10000 / alerts_wired_to_this_table) => alerts_below_the_interval_per_myriad

"refresh interval, seconds       : " + str(refresh_seconds) ^0
"query to pixel, ms              : " + str(query_to_pixel_ms) ^0
"timestamps shown on the page    : " + str(timestamps_shown_on_the_page) ^0
"  showing when data was produced: " + str(timestamps_showing_when_the_data_was_produced) ^0
"" ^0
"pipeline interval, minutes      : " + str(pipeline_interval_minutes) ^0
"pipeline runtime, minutes       : " + str(pipeline_runtime_minutes) ^0
"worst case data age, minutes    : " + str(worst_case_data_age_minutes) ^0
"  in seconds                    : " + str(worst_case_data_age_seconds) ^0
"  the displayed age is          : " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of it" ^0
"" ^0
"alerts wired to this table      : " + str(alerts_wired_to_this_table) ^0
"  window shorter than the interval : " + str(alerts_whose_window_is_shorter_than_the_interval) ^0
"  share                         : " + str(alerts_below_the_interval_per_myriad) + " per ten thousand" ^0
"" ^0

# ---- what the page verified ----

"the live page" ^0
"  transport : a push connection, not a poll" ^0
"  refresh interval, seconds : " + str(refresh_seconds) ^0
"  query to pixel, ms        : " + str(query_to_pixel_ms) ^0
"  does it show its own freshness : yes, a visible" ^0
"    timestamp rather than nothing" ^0
"  is that timestamp correct : exactly correct" ^0
"  verdict : LIVE" ^0
"" ^0
"  showing a last-updated time at all is more than most" ^0
"  dashboards do and the number in it is right" ^0
"" ^0

# ---- what it is the age of ----

"the timestamp" ^0
"  measures : the interval between now and the last refresh" ^0
"  what refreshed : the page" ^0
"  what did not refresh : the table, which the batch owns" ^0
"  when the batch last wrote : up to " + str(worst_case_data_age_minutes) + " minutes ago" ^0
"  a timestamp for that : " + str(timestamps_showing_when_the_data_was_produced) ^0
"  where the data would have to carry it : in the rows" ^0
"" ^0
"  the page is fresh with respect to the table and the" ^0
"  reader is asking about the world" ^0
"" ^0
# ---- the two intervals ----

# Both are real intervals and both are measured. One of them is the one the
# reader is deciding with, and it is the one nothing displays.
"the two clocks on this page" ^0
"  time since the page refreshed : " + str(refresh_seconds) + " seconds, displayed" ^0
"  time since the number was true : up to" ^0
"    " + str(worst_case_data_age_minutes) + " minutes, not displayed" ^0
"  ratio between them : the first is" ^0
"    " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of the second" ^0
"  which one the reader is shown : the first" ^0
"  which one the reader is using : the second" ^0
"" ^0

# ---- what a stalled pipeline looks like ----

# If the batch stops entirely the page keeps refreshing, keeps succeeding, and
# keeps saying it updated seconds ago. The failure mode of the thing that feeds
# it is invisible in the thing that displays it.
"the batch stops" ^0
"  page refreshes : continue" ^0
"  query errors   : none, the table is still there" ^0
"  timestamp shown : still seconds old" ^0
"  values shown    : the last batch's, indefinitely" ^0
"  what would change on the page : nothing" ^0
"  what would have to change for it to show : a timestamp" ^0
"    carried by the data rather than by the request" ^0
"" ^0

# ---- the four alerts ----

"the alerts on this table" ^0
"  wired to it : " + str(alerts_wired_to_this_table) ^0
"  with a window shorter than the interval : " + str(alerts_whose_window_is_shorter_than_the_interval) ^0
"  what a window shorter than the interval means : the" ^0
"    condition is evaluated over a period the source" ^0
"    cannot have observed separately" ^0
"  are those alerts firing wrongly : no; they fire late" ^0
"    and correctly, on the batch that carries the change" ^0
"  what the page implies about their latency : seconds" ^0
"" ^0

# ---- null control ----

# The same page, showing the maximum event timestamp present in the rows rather
# than the time of the request that fetched them.
worst_case_data_age_minutes => nc_age_the_page_displays_minutes
timestamps_shown_on_the_page => nc_timestamps_showing_when_the_data_was_produced

"null control - the page shows the data's own timestamp" ^0
"  refresh interval : " + str(refresh_seconds) + " seconds, unchanged" ^0
"  timestamps showing when data was produced : " + str(nc_timestamps_showing_when_the_data_was_produced) ^0
"  age the page displays, minutes : up to " + str(nc_age_the_page_displays_minutes) ^0
"  the page did not get slower or faster; it started" ^0
"  reporting the age of the answer instead of the age of" ^0
"  the question" ^0
"" ^0

# ---- the rule ----

"what a live dashboard guarantees" ^0
"  what you see is what the table holds, now : exactly," ^0
"    within " + str(query_to_pixel_ms) + " ms, over a push connection" ^0
"  what you see is what is happening now     : not" ^0
"    addressed; the page is one hop from the reader and" ^0
"    the batch is the hop that carries the age" ^0
"" ^0
"freshness composes along a chain and a display can only" ^0
"measure its own link; a timestamp attached to the fetch is" ^0
"a property of the fetch, and the one a reader needs has to" ^0
"be produced where the value was" ^0
"" ^0

"The page is genuinely live: a push connection, " + str(query_to_pixel_ms) + " ms from query to pixel, and a" ^0
"visible last-updated timestamp that is exactly correct about the refresh. The" ^0
"table under it is written by a batch every " + str(pipeline_interval_minutes) + " minutes taking " + str(pipeline_runtime_minutes) + ", so the" ^0
"displayed age is " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of the " + str(worst_case_data_age_minutes) + "-minute worst case, with" ^0
str(timestamps_showing_when_the_data_was_produced) + " timestamps for the data itself and " + str(alerts_whose_window_is_shorter_than_the_interval) + " of " + str(alerts_wired_to_this_table) + " alerts - " + str(alerts_below_the_interval_per_myriad) ^0
"per ten thousand - watching windows shorter than the interval that feeds them." ^0

refresh_seconds = 5
query_to_pixel_ms = 40
pipeline_interval_minutes = 60
pipeline_runtime_minutes = 14
seconds_in_a_minute = 60
timestamps_shown_on_the_page = 1
timestamps_showing_when_the_data_was_produced = 0
alerts_wired_to_this_table = 6
alerts_whose_window_is_shorter_than_the_interval = 4
worst_case_data_age_minutes = pipeline_interval_minutes + pipeline_runtime_minutes
worst_case_data_age_seconds = worst_case_data_age_minutes * seconds_in_a_minute
displayed_age_per_myriad_of_actual = int(refresh_seconds * 10000 / worst_case_data_age_seconds)
alerts_below_the_interval_per_myriad = int(alerts_whose_window_is_shorter_than_the_interval * 10000 / alerts_wired_to_this_table)
print("refresh interval, seconds       : " + str(refresh_seconds))
print("query to pixel, ms              : " + str(query_to_pixel_ms))
print("timestamps shown on the page    : " + str(timestamps_shown_on_the_page))
print("  showing when data was produced: " + str(timestamps_showing_when_the_data_was_produced))
print("")
print("pipeline interval, minutes      : " + str(pipeline_interval_minutes))
print("pipeline runtime, minutes       : " + str(pipeline_runtime_minutes))
print("worst case data age, minutes    : " + str(worst_case_data_age_minutes))
print("  in seconds                    : " + str(worst_case_data_age_seconds))
print("  the displayed age is          : " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of it")
print("")
print("alerts wired to this table      : " + str(alerts_wired_to_this_table))
print("  window shorter than the interval : " + str(alerts_whose_window_is_shorter_than_the_interval))
print("  share                         : " + str(alerts_below_the_interval_per_myriad) + " per ten thousand")
print("")
print("the live page")
print("  transport : a push connection, not a poll")
print("  refresh interval, seconds : " + str(refresh_seconds))
print("  query to pixel, ms        : " + str(query_to_pixel_ms))
print("  does it show its own freshness : yes, a visible")
print("    timestamp rather than nothing")
print("  is that timestamp correct : exactly correct")
print("  verdict : LIVE")
print("")
print("  showing a last-updated time at all is more than most")
print("  dashboards do and the number in it is right")
print("")
print("the timestamp")
print("  measures : the interval between now and the last refresh")
print("  what refreshed : the page")
print("  what did not refresh : the table, which the batch owns")
print("  when the batch last wrote : up to " + str(worst_case_data_age_minutes) + " minutes ago")
print("  a timestamp for that : " + str(timestamps_showing_when_the_data_was_produced))
print("  where the data would have to carry it : in the rows")
print("")
print("  the page is fresh with respect to the table and the")
print("  reader is asking about the world")
print("")
print("the two clocks on this page")
print("  time since the page refreshed : " + str(refresh_seconds) + " seconds, displayed")
print("  time since the number was true : up to")
print("    " + str(worst_case_data_age_minutes) + " minutes, not displayed")
print("  ratio between them : the first is")
print("    " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of the second")
print("  which one the reader is shown : the first")
print("  which one the reader is using : the second")
print("")
print("the batch stops")
print("  page refreshes : continue")
print("  query errors   : none, the table is still there")
print("  timestamp shown : still seconds old")
print("  values shown    : the last batch's, indefinitely")
print("  what would change on the page : nothing")
print("  what would have to change for it to show : a timestamp")
print("    carried by the data rather than by the request")
print("")
print("the alerts on this table")
print("  wired to it : " + str(alerts_wired_to_this_table))
print("  with a window shorter than the interval : " + str(alerts_whose_window_is_shorter_than_the_interval))
print("  what a window shorter than the interval means : the")
print("    condition is evaluated over a period the source")
print("    cannot have observed separately")
print("  are those alerts firing wrongly : no; they fire late")
print("    and correctly, on the batch that carries the change")
print("  what the page implies about their latency : seconds")
print("")
nc_age_the_page_displays_minutes = worst_case_data_age_minutes
nc_timestamps_showing_when_the_data_was_produced = timestamps_shown_on_the_page
print("null control - the page shows the data's own timestamp")
print("  refresh interval : " + str(refresh_seconds) + " seconds, unchanged")
print("  timestamps showing when data was produced : " + str(nc_timestamps_showing_when_the_data_was_produced))
print("  age the page displays, minutes : up to " + str(nc_age_the_page_displays_minutes))
print("  the page did not get slower or faster; it started")
print("  reporting the age of the answer instead of the age of")
print("  the question")
print("")
print("what a live dashboard guarantees")
print("  what you see is what the table holds, now : exactly,")
print("    within " + str(query_to_pixel_ms) + " ms, over a push connection")
print("  what you see is what is happening now     : not")
print("    addressed; the page is one hop from the reader and")
print("    the batch is the hop that carries the age")
print("")
print("freshness composes along a chain and a display can only")
print("measure its own link; a timestamp attached to the fetch is")
print("a property of the fetch, and the one a reader needs has to")
print("be produced where the value was")
print("")
print("The page is genuinely live: a push connection, " + str(query_to_pixel_ms) + " ms from query to pixel, and a")
print("visible last-updated timestamp that is exactly correct about the refresh. The")
print("table under it is written by a batch every " + str(pipeline_interval_minutes) + " minutes taking " + str(pipeline_runtime_minutes) + ", so the")
print("displayed age is " + str(displayed_age_per_myriad_of_actual) + " per ten thousand of the " + str(worst_case_data_age_minutes) + "-minute worst case, with")
print(str(timestamps_showing_when_the_data_was_produced) + " timestamps for the data itself and " + str(alerts_whose_window_is_shorter_than_the_interval) + " of " + str(alerts_wired_to_this_table) + " alerts - " + str(alerts_below_the_interval_per_myriad))
print("per ten thousand - watching windows shorter than the interval that feeds them.")

refresh interval, seconds       : 5
query to pixel, ms              : 40
timestamps shown on the page    : 1
  showing when data was produced: 0

pipeline interval, minutes      : 60
pipeline runtime, minutes       : 14
worst case data age, minutes    : 74
  in seconds                    : 4440
  the displayed age is          : 11 per ten thousand of it

alerts wired to this table      : 6
  window shorter than the interval : 4
  share                         : 6666 per ten thousand

the live page
  transport : a push connection, not a poll
  refresh interval, seconds : 5
  query to pixel, ms        : 40
  does it show its own freshness : yes, a visible
    timestamp rather than nothing
  is that timestamp correct : exactly correct
  verdict : LIVE

  showing a last-updated time at all is more than most
  dashboards do and the number in it is right

the timestamp
  measures : the interval between now and the last refresh
  what refreshed : the page
  what did not refresh : the table, which the batch owns
  when the batch last wrote : up to 74 minutes ago
  a timestamp for that : 0
  where the data would have to carry it : in the rows

  the page is fresh with respect to the table and the
  reader is asking about the world

the two clocks on this page
  time since the page refreshed : 5 seconds, displayed
  time since the number was true : up to
    74 minutes, not displayed
  ratio between them : the first is
    11 per ten thousand of the second
  which one the reader is shown : the first
  which one the reader is using : the second

the batch stops
  page refreshes : continue
  query errors   : none, the table is still there
  timestamp shown : still seconds old
  values shown    : the last batch's, indefinitely
  what would change on the page : nothing
  what would have to change for it to show : a timestamp
    carried by the data rather than by the request

the alerts on this table
  wired to it : 6
  with a window shorter than the interval : 4
  what a window shorter than the interval means : the
    condition is evaluated over a period the source
    cannot have observed separately
  are those alerts firing wrongly : no; they fire late
    and correctly, on the batch that carries the change
  what the page implies about their latency : seconds

null control - the page shows the data's own timestamp
  refresh interval : 5 seconds, unchanged
  timestamps showing when data was produced : 1
  age the page displays, minutes : up to 74
  the page did not get slower or faster; it started
  reporting the age of the answer instead of the age of
  the question

what a live dashboard guarantees
  what you see is what the table holds, now : exactly,
    within 40 ms, over a push connection
  what you see is what is happening now     : not
    addressed; the page is one hop from the reader and
    the batch is the hop that carries the age

freshness composes along a chain and a display can only
measure its own link; a timestamp attached to the fetch is
a property of the fetch, and the one a reader needs has to
be produced where the value was

The page is genuinely live: a push connection, 40 ms from query to pixel, and a
visible last-updated timestamp that is exactly correct about the refresh. The
table under it is written by a batch every 60 minutes taking 14, so the
displayed age is 11 per ten thousand of the 74-minute worst case, with
0 timestamps for the data itself and 4 of 6 alerts - 6666
per ten thousand - watching windows shorter than the interval that feeds them.

Trace event types