The same rule blocks here and warns there

# Self-authored for the EML case corpus (no external origin). One policy, six
# pipelines, three enforcement levels. Where the violations end up is computed
# below.
#
# The graduated rollout was right and is what everybody recommends. A rule that
# blocks on day one blocks the teams who had no warning, so it starts as a
# warning, becomes a soft failure, and blocks last. Each team got time
# proportional to how much work the rule created for it.
#
# What was graduated by intent was the schedule. What it produced is a standing
# difference in what the rule means per pipeline, because the last step never
# happened for the pipelines where it was hardest - which are the pipelines with
# the most violations.
#
# Violations are counted per pipeline at each enforcement level.

# [pipeline, enforcement level (0 warn, 1 soft-fail, 2 block), violations per week, months at this level]
[["web", 2, 0, 14], ["mobile", 2, 0, 12], ["internal tools", 1, 3, 11], ["data pipeline", 0, 26, 13], ["legacy billing", 0, 41, 13], ["partner sync", 1, 7, 9]] => pipes

len(pipes) => n
["warns", "soft-fails", "blocks"] => levels

0 => total_v
0 => at_block
0 => at_warn
for p in pipes:
    total_v + p[2] => total_v
    if p[1] == 2:
        at_block + p[2] => at_block
    if p[1] == 0:
        at_warn + p[2] => at_warn

"pipelines : " + str(n) ^0
"violations reaching production per week : " + str(total_v) ^0
"" ^0
"pipeline          enforcement   violations/week   months at this level" ^0
for p in pipes:
    "  " + p[0] + "   " + levels[p[1]] + "        " + str(p[2]) + "                " + str(p[3]) ^0
"" ^0

"by enforcement level" ^0
for k in [0:2]:
    0 => c
    0 => v
    for p in pipes:
        if p[1] == k:
            c + 1 => c
            v + p[2] => v
    "  " + levels[k] + " : " + str(c) + " pipeline(s), " + str(v) + " violations/week" ^0
"" ^0
if at_warn > at_block:
    "the pipelines that only warn carry " + str(at_warn) + " of the " + str(total_v) + " violations" ^0
    "  which is " + str(int(at_warn * 100 / total_v)) + "%" ^0
"" ^0

# ---- the graduation never finished, and where it stopped is not random ----

0 => stalled
for p in pipes:
    if p[1] < 2:
        stalled + 1 => stalled
"pipelines still below full enforcement : " + str(stalled) + " of " + str(n) ^0
0 => longest
for p in pipes:
    if p[3] > longest:
        p[3] => longest
"months since the rollout began : " + str(longest) ^0
"  the schedule had three steps and no step has moved for the stalled ones" ^0
"" ^0

"why each one stalled, stated as its violation count" ^0
for p in pipes:
    if p[1] < 2:
        "  " + p[0] + " : " + str(p[2]) + " violations a week would have to be fixed first" ^0
"  the cost of advancing is proportional to the violations, so the pipelines" ^0
"  furthest from compliance are the ones it is most expensive to advance" ^0
"" ^0

# ---- what the policy dashboard says ----

"the rollout, as reported" ^0
"  pipelines the policy is applied to : " + str(n) + " of " + str(n) + ", which is 100%" ^0
"  pipelines where a violation cannot reach production : " + str(n - stalled) ^0
if stalled > 0:
    "  the first number is what gets reported and the second is what the rule" ^0
    "  does" ^0
"" ^0

# ---- what the rule means to each team ----

"what a developer experiences" ^0
for p in pipes:
    "" => exp
    if p[1] == 2:
        "cannot merge" => exp
    elif p[1] == 1:
        "a red check they can override" => exp
    else:
        "a line in a log" => exp
    "  " + p[0] + " : " + exp ^0
"  the same policy document produces three different experiences, and which" ^0
"  one a developer gets is decided by which repository they opened" ^0
"" ^0

# ---- what advancing one level would cost and buy ----

0 => next_step_fixes
for p in pipes:
    if p[1] == 0:
        next_step_fixes + p[2] => next_step_fixes
"advancing the two warn-only pipelines to soft-fail" ^0
"  violations that would start being surfaced : " + str(next_step_fixes) + " a week" ^0
"  violations that would be blocked           : 0, soft-fail is overridable" ^0
"  what it changes is who sees them, which is the step that was skipped" ^0
"" ^0

# ---- the control: a rule that shipped at full enforcement ----
#
# Where the rule blocked everywhere from the first day, the enforcement level
# is one number rather than six, and no pipeline can drift from it.

"control - a policy that blocked on every pipeline from day one" ^0
"  enforcement levels in play : 1" ^0
"  pipelines that can accumulate violations : 0" ^0
"  the cost was paid at once by the teams with the most work, which is the" ^0
"  cost the graduated rollout was designed to spread - and spreading it is" ^0
"  what left it unpaid" ^0
"" ^0

"The graduated rollout gave each team time proportional to its work, which is" ^0
"the fair schedule. The last step is hardest exactly where there is most to" ^0
"fix, so it is unfinished where it would matter most." ^0

pipes = [["web", 2, 0, 14], ["mobile", 2, 0, 12], ["internal tools", 1, 3, 11], ["data pipeline", 0, 26, 13], ["legacy billing", 0, 41, 13], ["partner sync", 1, 7, 9]]
n = len(pipes)
levels = ["warns", "soft-fails", "blocks"]
total_v = 0
at_block = 0
at_warn = 0
for p in pipes:
    total_v = total_v + p[2]
    if p[1] == 2:
        at_block = at_block + p[2]
    if p[1] == 0:
        at_warn = at_warn + p[2]
print("pipelines : " + str(n))
print("violations reaching production per week : " + str(total_v))
print("")
print("pipeline          enforcement   violations/week   months at this level")
for p in pipes:
    print("  " + p[0] + "   " + levels[p[1]] + "        " + str(p[2]) + "                " + str(p[3]))
print("")
print("by enforcement level")
for k in range(0, 3):
    c = 0
    v = 0
    for p in pipes:
        if p[1] == k:
            c = c + 1
            v = v + p[2]
    print("  " + levels[k] + " : " + str(c) + " pipeline(s), " + str(v) + " violations/week")
print("")
if at_warn > at_block:
    print("the pipelines that only warn carry " + str(at_warn) + " of the " + str(total_v) + " violations")
    print("  which is " + str(int(at_warn * 100 / total_v)) + "%")
print("")
stalled = 0
for p in pipes:
    if p[1] < 2:
        stalled = stalled + 1
print("pipelines still below full enforcement : " + str(stalled) + " of " + str(n))
longest = 0
for p in pipes:
    if p[3] > longest:
        longest = p[3]
print("months since the rollout began : " + str(longest))
print("  the schedule had three steps and no step has moved for the stalled ones")
print("")
print("why each one stalled, stated as its violation count")
for p in pipes:
    if p[1] < 2:
        print("  " + p[0] + " : " + str(p[2]) + " violations a week would have to be fixed first")
print("  the cost of advancing is proportional to the violations, so the pipelines")
print("  furthest from compliance are the ones it is most expensive to advance")
print("")
print("the rollout, as reported")
print("  pipelines the policy is applied to : " + str(n) + " of " + str(n) + ", which is 100%")
print("  pipelines where a violation cannot reach production : " + str(n - stalled))
if stalled > 0:
    print("  the first number is what gets reported and the second is what the rule")
    print("  does")
print("")
print("what a developer experiences")
for p in pipes:
    exp = ""
    if p[1] == 2:
        exp = "cannot merge"
    elif p[1] == 1:
        exp = "a red check they can override"
    else:
        exp = "a line in a log"
    print("  " + p[0] + " : " + exp)
print("  the same policy document produces three different experiences, and which")
print("  one a developer gets is decided by which repository they opened")
print("")
next_step_fixes = 0
for p in pipes:
    if p[1] == 0:
        next_step_fixes = next_step_fixes + p[2]
print("advancing the two warn-only pipelines to soft-fail")
print("  violations that would start being surfaced : " + str(next_step_fixes) + " a week")
print("  violations that would be blocked           : 0, soft-fail is overridable")
print("  what it changes is who sees them, which is the step that was skipped")
print("")
print("control - a policy that blocked on every pipeline from day one")
print("  enforcement levels in play : 1")
print("  pipelines that can accumulate violations : 0")
print("  the cost was paid at once by the teams with the most work, which is the")
print("  cost the graduated rollout was designed to spread - and spreading it is")
print("  what left it unpaid")
print("")
print("The graduated rollout gave each team time proportional to its work, which is")
print("the fair schedule. The last step is hardest exactly where there is most to")
print("fix, so it is unfinished where it would matter most.")

pipelines : 6
violations reaching production per week : 77

pipeline          enforcement   violations/week   months at this level
  web   blocks        0                14
  mobile   blocks        0                12
  internal tools   soft-fails        3                11
  data pipeline   warns        26                13
  legacy billing   warns        41                13
  partner sync   soft-fails        7                9

by enforcement level
  warns : 2 pipeline(s), 67 violations/week
  soft-fails : 2 pipeline(s), 10 violations/week
  blocks : 2 pipeline(s), 0 violations/week

the pipelines that only warn carry 67 of the 77 violations
  which is 87%

pipelines still below full enforcement : 4 of 6
months since the rollout began : 14
  the schedule had three steps and no step has moved for the stalled ones

why each one stalled, stated as its violation count
  internal tools : 3 violations a week would have to be fixed first
  data pipeline : 26 violations a week would have to be fixed first
  legacy billing : 41 violations a week would have to be fixed first
  partner sync : 7 violations a week would have to be fixed first
  the cost of advancing is proportional to the violations, so the pipelines
  furthest from compliance are the ones it is most expensive to advance

the rollout, as reported
  pipelines the policy is applied to : 6 of 6, which is 100%
  pipelines where a violation cannot reach production : 2
  the first number is what gets reported and the second is what the rule
  does

what a developer experiences
  web : cannot merge
  mobile : cannot merge
  internal tools : a red check they can override
  data pipeline : a line in a log
  legacy billing : a line in a log
  partner sync : a red check they can override
  the same policy document produces three different experiences, and which
  one a developer gets is decided by which repository they opened

advancing the two warn-only pipelines to soft-fail
  violations that would start being surfaced : 67 a week
  violations that would be blocked           : 0, soft-fail is overridable
  what it changes is who sees them, which is the step that was skipped

control - a policy that blocked on every pipeline from day one
  enforcement levels in play : 1
  pipelines that can accumulate violations : 0
  the cost was paid at once by the teams with the most work, which is the
  cost the graduated rollout was designed to spread - and spreading it is
  what left it unpaid

The graduated rollout gave each team time proportional to its work, which is
the fair schedule. The last step is hardest exactly where there is most to
fix, so it is unfinished where it would matter most.

Trace event types