The thirty percent forecast was judged on one rainy day

# Self-authored for the EML case corpus (no external origin). The forecast said
# thirty percent chance of rain, it rained, and the forecast is judged wrong.
# The forecast was real, the rain was real, and the judgement compared the two
# exactly. What a probability can be judged against is computed below.
#
# The judgement is careful. It reads the real forecast and the real outcome; it
# compares them for the actual day; the rain is not in dispute; and the intent
# is exactly 'was the forecast right'.
#
# A thirty-percent forecast is a claim about many days, not one: it is right if
# it rains on about thirty of every hundred such days, and a single rainy day is
# one of the thirty it promised.

1000 => days_forecast_at_thirty_percent
300 => of_those_days_it_rained
3000 => forecast_per_myriad
1 => rainy_days_used_for_the_judgement

int(of_those_days_it_rained * 10000 / days_forecast_at_thirty_percent) => observed_rain_rate_per_myriad
observed_rain_rate_per_myriad - forecast_per_myriad => calibration_gap_per_myriad
days_forecast_at_thirty_percent - of_those_days_it_rained => dry_days_the_forecast_also_covered
int(rainy_days_used_for_the_judgement * 10000 / days_forecast_at_thirty_percent) => share_of_the_evidence_the_judgement_used_per_myriad

"forecast                        : " + str(forecast_per_myriad) + " per ten thousand chance of rain" ^0
"days carrying that forecast     : " + str(days_forecast_at_thirty_percent) ^0
"of those, days it rained        : " + str(of_those_days_it_rained) ^0
"observed rain rate              : " + str(observed_rain_rate_per_myriad) + " per ten thousand" ^0
"calibration gap                 : " + str(calibration_gap_per_myriad) ^0
"" ^0
"days used for the judgement     : " + str(rainy_days_used_for_the_judgement) + ", a rainy one" ^0
"share of the evidence used      : " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand" ^0
"dry days the same forecast covered : " + str(dry_days_the_forecast_also_covered) ^0
"" ^0

# ---- what the judgement verified ----

"the one-day judgement" ^0
"  forecast : the real one, thirty percent" ^0
"  outcome : the real one, it rained" ^0
"  comparison : for the actual day" ^0
"  intent : was the forecast right" ^0
"  facts wrong : 0" ^0
"  verdict : IT SAID THIRTY AND IT RAINED, SO IT WAS WRONG" ^0
"" ^0
"  reading the real forecast and the real outcome is the" ^0
"  part done right here, and it is why nothing about the" ^0
"  day itself is in dispute" ^0
"" ^0

# ---- what a probability is a claim about ----

"calibration" ^0
"  what thirty percent promises : rain on about thirty of" ^0
"    every hundred such days" ^0
"  what one rainy day is : one of those thirty" ^0
"  what one dry day is : one of the seventy" ^0
"  so a single day : cannot confirm or refute the number" ^0
"  what can : the " + str(days_forecast_at_thirty_percent) + " such days together, which rained " + str(observed_rain_rate_per_myriad) ^0
"    per ten thousand of the time - exactly as promised" ^0
"" ^0

# ---- what the forecaster got ----

"the verdict" ^0
"  judged : wrong, on one day" ^0
"  measured : calibrated, over a thousand days" ^0
"  is the outcome misrecorded : no" ^0
"  is a probability a prediction of one day : no; it is a" ^0
"    rate over the days it is said on" ^0
"" ^0

# ---- null control ----

# The same forecast scored on all the days it was issued (a calibration
# check), not on the one day that happened to be rainy.
0 => nc_days_that_can_refute_a_rate
3000 => nc_observed_rate_across_all_thirty_percent_days_per_myriad
1 => nc_forecast_judged_calibrated

"null control - score the forecast over every day it was issued" ^0
"  single days that can refute a rate : " + str(nc_days_that_can_refute_a_rate) ^0
"  observed rate across all thirty-percent days : " + str(nc_observed_rate_across_all_thirty_percent_days_per_myriad) + " per ten thousand" ^0
"  forecast judged calibrated : " + str(nc_forecast_judged_calibrated) ^0
"  no forecast and no weather changed; the judgement stopped" ^0
"  using " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand of the evidence" ^0
"" ^0

# ---- the rule ----

"what a one-day comparison guarantees" ^0
"  on that day the forecast said thirty and it rained :" ^0
"    exactly, real forecast, real outcome" ^0
"  the forecast was wrong : not addressed; thirty percent is" ^0
"    a rate over the days it is issued, and across " + str(days_forecast_at_thirty_percent) + " such" ^0
"    days it rained " + str(observed_rain_rate_per_myriad) + " per ten thousand of the time - the rainy" ^0
"    day was one of the promised " + str(of_those_days_it_rained) ^0
"" ^0

"a probability is a promise about a population of days, and a single day is a" ^0
"member of that population, not a test of it; the only thing one outcome can" ^0
"refute is a forecast of zero or of one" ^0
"" ^0

"The forecast really said thirty and it really rained - the day is not in" ^0
"dispute. But thirty percent is a rate: across " + str(days_forecast_at_thirty_percent) + " days with that forecast it" ^0
"rained " + str(of_those_days_it_rained) + " times, " + str(observed_rain_rate_per_myriad) + " per ten thousand, exactly as promised; the judgement" ^0
"used " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand of the evidence, until the forecast was scored on all of it." ^0

days_forecast_at_thirty_percent = 1000
of_those_days_it_rained = 300
forecast_per_myriad = 3000
rainy_days_used_for_the_judgement = 1
observed_rain_rate_per_myriad = int(of_those_days_it_rained * 10000 / days_forecast_at_thirty_percent)
calibration_gap_per_myriad = observed_rain_rate_per_myriad - forecast_per_myriad
dry_days_the_forecast_also_covered = days_forecast_at_thirty_percent - of_those_days_it_rained
share_of_the_evidence_the_judgement_used_per_myriad = int(rainy_days_used_for_the_judgement * 10000 / days_forecast_at_thirty_percent)
print("forecast                        : " + str(forecast_per_myriad) + " per ten thousand chance of rain")
print("days carrying that forecast     : " + str(days_forecast_at_thirty_percent))
print("of those, days it rained        : " + str(of_those_days_it_rained))
print("observed rain rate              : " + str(observed_rain_rate_per_myriad) + " per ten thousand")
print("calibration gap                 : " + str(calibration_gap_per_myriad))
print("")
print("days used for the judgement     : " + str(rainy_days_used_for_the_judgement) + ", a rainy one")
print("share of the evidence used      : " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand")
print("dry days the same forecast covered : " + str(dry_days_the_forecast_also_covered))
print("")
print("the one-day judgement")
print("  forecast : the real one, thirty percent")
print("  outcome : the real one, it rained")
print("  comparison : for the actual day")
print("  intent : was the forecast right")
print("  facts wrong : 0")
print("  verdict : IT SAID THIRTY AND IT RAINED, SO IT WAS WRONG")
print("")
print("  reading the real forecast and the real outcome is the")
print("  part done right here, and it is why nothing about the")
print("  day itself is in dispute")
print("")
print("calibration")
print("  what thirty percent promises : rain on about thirty of")
print("    every hundred such days")
print("  what one rainy day is : one of those thirty")
print("  what one dry day is : one of the seventy")
print("  so a single day : cannot confirm or refute the number")
print("  what can : the " + str(days_forecast_at_thirty_percent) + " such days together, which rained " + str(observed_rain_rate_per_myriad))
print("    per ten thousand of the time - exactly as promised")
print("")
print("the verdict")
print("  judged : wrong, on one day")
print("  measured : calibrated, over a thousand days")
print("  is the outcome misrecorded : no")
print("  is a probability a prediction of one day : no; it is a")
print("    rate over the days it is said on")
print("")
nc_days_that_can_refute_a_rate = 0
nc_observed_rate_across_all_thirty_percent_days_per_myriad = 3000
nc_forecast_judged_calibrated = 1
print("null control - score the forecast over every day it was issued")
print("  single days that can refute a rate : " + str(nc_days_that_can_refute_a_rate))
print("  observed rate across all thirty-percent days : " + str(nc_observed_rate_across_all_thirty_percent_days_per_myriad) + " per ten thousand")
print("  forecast judged calibrated : " + str(nc_forecast_judged_calibrated))
print("  no forecast and no weather changed; the judgement stopped")
print("  using " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand of the evidence")
print("")
print("what a one-day comparison guarantees")
print("  on that day the forecast said thirty and it rained :")
print("    exactly, real forecast, real outcome")
print("  the forecast was wrong : not addressed; thirty percent is")
print("    a rate over the days it is issued, and across " + str(days_forecast_at_thirty_percent) + " such")
print("    days it rained " + str(observed_rain_rate_per_myriad) + " per ten thousand of the time - the rainy")
print("    day was one of the promised " + str(of_those_days_it_rained))
print("")
print("a probability is a promise about a population of days, and a single day is a")
print("member of that population, not a test of it; the only thing one outcome can")
print("refute is a forecast of zero or of one")
print("")
print("The forecast really said thirty and it really rained - the day is not in")
print("dispute. But thirty percent is a rate: across " + str(days_forecast_at_thirty_percent) + " days with that forecast it")
print("rained " + str(of_those_days_it_rained) + " times, " + str(observed_rain_rate_per_myriad) + " per ten thousand, exactly as promised; the judgement")
print("used " + str(share_of_the_evidence_the_judgement_used_per_myriad) + " per ten thousand of the evidence, until the forecast was scored on all of it.")

forecast                        : 3000 per ten thousand chance of rain
days carrying that forecast     : 1000
of those, days it rained        : 300
observed rain rate              : 3000 per ten thousand
calibration gap                 : 0

days used for the judgement     : 1, a rainy one
share of the evidence used      : 10 per ten thousand
dry days the same forecast covered : 700

the one-day judgement
  forecast : the real one, thirty percent
  outcome : the real one, it rained
  comparison : for the actual day
  intent : was the forecast right
  facts wrong : 0
  verdict : IT SAID THIRTY AND IT RAINED, SO IT WAS WRONG

  reading the real forecast and the real outcome is the
  part done right here, and it is why nothing about the
  day itself is in dispute

calibration
  what thirty percent promises : rain on about thirty of
    every hundred such days
  what one rainy day is : one of those thirty
  what one dry day is : one of the seventy
  so a single day : cannot confirm or refute the number
  what can : the 1000 such days together, which rained 3000
    per ten thousand of the time - exactly as promised

the verdict
  judged : wrong, on one day
  measured : calibrated, over a thousand days
  is the outcome misrecorded : no
  is a probability a prediction of one day : no; it is a
    rate over the days it is said on

null control - score the forecast over every day it was issued
  single days that can refute a rate : 0
  observed rate across all thirty-percent days : 3000 per ten thousand
  forecast judged calibrated : 1
  no forecast and no weather changed; the judgement stopped
  using 10 per ten thousand of the evidence

what a one-day comparison guarantees
  on that day the forecast said thirty and it rained :
    exactly, real forecast, real outcome
  the forecast was wrong : not addressed; thirty percent is
    a rate over the days it is issued, and across 1000 such
    days it rained 3000 per ten thousand of the time - the rainy
    day was one of the promised 300

a probability is a promise about a population of days, and a single day is a
member of that population, not a test of it; the only thing one outcome can
refute is a forecast of zero or of one

The forecast really said thirty and it really rained - the day is not in
dispute. But thirty percent is a rate: across 1000 days with that forecast it
rained 300 times, 3000 per ten thousand, exactly as promised; the judgement
used 10 per ten thousand of the evidence, until the forecast was scored on all of it.

Trace event types