The review caught the bug and the revert was partial

# Self-authored for the EML case corpus (no external origin). A reviewer found
# the defect, the author agreed, and the commit was reverted cleanly before
# release. What the revert removed is computed below.
#
# The review worked exactly as it is supposed to. A human reading a diff
# noticed that a comparison was wrong, said so, and was right; the author
# agreed within the hour; the offending commit was reverted; the revert applied
# with no conflict; a second reviewer confirmed the reverting diff was the
# exact inverse of the original; and the full suite was green afterwards. Three
# hundred forty defects were caught this way this year.
#
# A revert is the inverse of a DIFF. It restores the lines that commit changed,
# and in the twenty-six days between the commit and the review the function had
# been extracted and called from two more places.
#
# The revert removed one of the three call sites.

340 => defects_caught_in_review_this_year
0 => conflicts_when_the_revert_applied
1 => second_reviewers_confirming_the_inverse
0 => failing_tests_after_the_revert
26 => days_between_the_commit_and_the_review
14 => commits_touching_the_file_in_between
1 => call_sites_when_the_defect_was_introduced
3 => call_sites_when_the_revert_was_written
1 => call_sites_the_revert_removed
0 => tests_asserting_the_defective_behaviour_is_gone

call_sites_when_the_revert_was_written - call_sites_the_revert_removed => call_sites_remaining
int(call_sites_remaining * 10000 / call_sites_when_the_revert_was_written) => remaining_per_myriad

"defects caught in review this year : " + str(defects_caught_in_review_this_year) ^0
"conflicts when the revert applied  : " + str(conflicts_when_the_revert_applied) ^0
"second reviewers confirming the inverse : " + str(second_reviewers_confirming_the_inverse) ^0
"failing tests after the revert     : " + str(failing_tests_after_the_revert) ^0
"" ^0
"days between the commit and the review : " + str(days_between_the_commit_and_the_review) ^0
"commits touching the file in between   : " + str(commits_touching_the_file_in_between) ^0
"" ^0
"call sites when the defect was introduced : " + str(call_sites_when_the_defect_was_introduced) ^0
"call sites when the revert was written    : " + str(call_sites_when_the_revert_was_written) ^0
"  removed by the revert                   : " + str(call_sites_the_revert_removed) ^0
"  remaining                               : " + str(call_sites_remaining) ^0
"  share remaining                         : " + str(remaining_per_myriad) + " per ten thousand" ^0
"tests asserting the behaviour is gone     : " + str(tests_asserting_the_defective_behaviour_is_gone) ^0
"" ^0

# ---- what the review verified ----

"the review" ^0
"  found by : a person reading the diff" ^0
"  the finding : a comparison that was wrong" ^0
"  was it right : yes, and the author agreed within the hour" ^0
"  defects caught this way this year : " + str(defects_caught_in_review_this_year) ^0
"  verdict : CAUGHT" ^0
"" ^0
"  a reviewer who reads carefully enough to find a wrong" ^0
"  comparison is the most valuable thing in this list" ^0
"" ^0

# ---- what the revert verified ----

"the revert" ^0
"  applied with conflicts : " + str(conflicts_when_the_revert_applied) ^0
"  confirmed as the exact inverse of the original diff : by" ^0
"    " + str(second_reviewers_confirming_the_inverse) + " second reviewer" ^0
"  suite after it : " + str(failing_tests_after_the_revert) + " failing" ^0
"  what it restored : the lines that commit changed" ^0
"  verdict : INVERTED" ^0
"" ^0
"  every one of those checks passed, and each of them is a" ^0
"  statement about the diff" ^0
"" ^0
# ---- what happened in twenty-six days ----

# The function was extracted during a refactor and called from two more places.
# Both later commits are good commits. Neither knew it was copying a defect,
# because the defect had not been found yet.
"the file between the commit and the review" ^0
"  commits touching it : " + str(commits_touching_the_file_in_between) ^0
"  the logic was       : extracted into a function" ^0
"  callers added       : two, in ordinary feature work" ^0
"  did either author know : no; nobody knew yet" ^0
"  call sites at review time : " + str(call_sites_when_the_revert_was_written) ^0
"" ^0

# ---- why the revert applied cleanly ----

# A clean apply means the lines the commit touched are still recognisable. It
# does not mean they are the only lines with that behaviour. The two copies are
# in files the original commit never touched, so there is nothing to conflict
# with.
"the clean apply" ^0
"  what a conflict would indicate : the original lines moved" ^0
"  what a clean apply indicates   : they did not" ^0
"  what either indicates about copies elsewhere : nothing" ^0
"  files the original commit touched : the one it touched" ^0
"  files the copies live in : two others" ^0
"" ^0

# ---- why the suite stayed green ----

# The suite was green before the defective commit and green after it, which is
# why review caught it rather than a test. Reverting one of three call sites
# changes nothing a test asserts.
"the suite" ^0
"  green before the defect : yes" ^0
"  green after the defect  : yes, which is why a human" ^0
"    found it and not a test" ^0
"  green after the revert  : yes" ^0
"  tests asserting the behaviour is gone : " + str(tests_asserting_the_defective_behaviour_is_gone) ^0
"  so the suite is measuring : that nothing else broke" ^0
"" ^0

# ---- null control ----

# The same finding, written down as a failing test first: a test that asserts
# the correct comparison, added before any code changes, and the fix is
# whatever makes it pass.
call_sites_when_the_revert_was_written => nc_call_sites_the_fix_removed
0 => nc_call_sites_remaining
1 => nc_tests_asserting_the_defective_behaviour_is_gone

"null control - the finding is written as a failing test" ^0
"  defect found by review : the same reviewer, unchanged" ^0
"  tests asserting the behaviour is gone : " + str(nc_tests_asserting_the_defective_behaviour_is_gone) ^0
"  call sites the fix must reach : " + str(nc_call_sites_the_fix_removed) ^0
"  call sites remaining : " + str(nc_call_sites_remaining) ^0
"  the review did not get better; the finding stopped being" ^0
"  addressed to a commit and started being addressed to a" ^0
"  behaviour" ^0
"" ^0

# ---- the rule ----

"what a clean revert guarantees" ^0
"  the tree no longer contains that commit's changes :" ^0
"    exactly, and it was confirmed line by line" ^0
"  the defect is gone                                : not" ^0
"    addressed; a commit is a location in a history and a" ^0
"    defect is a behaviour that can be copied" ^0
"" ^0
"a fix aimed at a commit is aimed at the place the defect was" ^0
"introduced; between introduction and discovery the code" ^0
"moves, and only a test names the behaviour rather than the" ^0
"place" ^0
"" ^0

"The review did the hard part: a person read the diff, found a wrong comparison," ^0
"was right, and " + str(defects_caught_in_review_this_year) + " defects were caught this way this year. The revert" ^0
"applied with " + str(conflicts_when_the_revert_applied) + " conflicts, was confirmed as the exact inverse of the original," ^0
"and left " + str(failing_tests_after_the_revert) + " tests failing - and in the " + str(days_between_the_commit_and_the_review) + " days and " + str(commits_touching_the_file_in_between) + " commits since, the logic" ^0
"had been extracted, so " + str(call_sites_remaining) + " of " + str(call_sites_when_the_revert_was_written) + " call sites - " + str(remaining_per_myriad) + " per ten thousand -" ^0
"survived, with " + str(tests_asserting_the_defective_behaviour_is_gone) + " tests naming the behaviour." ^0

defects_caught_in_review_this_year = 340
conflicts_when_the_revert_applied = 0
second_reviewers_confirming_the_inverse = 1
failing_tests_after_the_revert = 0
days_between_the_commit_and_the_review = 26
commits_touching_the_file_in_between = 14
call_sites_when_the_defect_was_introduced = 1
call_sites_when_the_revert_was_written = 3
call_sites_the_revert_removed = 1
tests_asserting_the_defective_behaviour_is_gone = 0
call_sites_remaining = call_sites_when_the_revert_was_written - call_sites_the_revert_removed
remaining_per_myriad = int(call_sites_remaining * 10000 / call_sites_when_the_revert_was_written)
print("defects caught in review this year : " + str(defects_caught_in_review_this_year))
print("conflicts when the revert applied  : " + str(conflicts_when_the_revert_applied))
print("second reviewers confirming the inverse : " + str(second_reviewers_confirming_the_inverse))
print("failing tests after the revert     : " + str(failing_tests_after_the_revert))
print("")
print("days between the commit and the review : " + str(days_between_the_commit_and_the_review))
print("commits touching the file in between   : " + str(commits_touching_the_file_in_between))
print("")
print("call sites when the defect was introduced : " + str(call_sites_when_the_defect_was_introduced))
print("call sites when the revert was written    : " + str(call_sites_when_the_revert_was_written))
print("  removed by the revert                   : " + str(call_sites_the_revert_removed))
print("  remaining                               : " + str(call_sites_remaining))
print("  share remaining                         : " + str(remaining_per_myriad) + " per ten thousand")
print("tests asserting the behaviour is gone     : " + str(tests_asserting_the_defective_behaviour_is_gone))
print("")
print("the review")
print("  found by : a person reading the diff")
print("  the finding : a comparison that was wrong")
print("  was it right : yes, and the author agreed within the hour")
print("  defects caught this way this year : " + str(defects_caught_in_review_this_year))
print("  verdict : CAUGHT")
print("")
print("  a reviewer who reads carefully enough to find a wrong")
print("  comparison is the most valuable thing in this list")
print("")
print("the revert")
print("  applied with conflicts : " + str(conflicts_when_the_revert_applied))
print("  confirmed as the exact inverse of the original diff : by")
print("    " + str(second_reviewers_confirming_the_inverse) + " second reviewer")
print("  suite after it : " + str(failing_tests_after_the_revert) + " failing")
print("  what it restored : the lines that commit changed")
print("  verdict : INVERTED")
print("")
print("  every one of those checks passed, and each of them is a")
print("  statement about the diff")
print("")
print("the file between the commit and the review")
print("  commits touching it : " + str(commits_touching_the_file_in_between))
print("  the logic was       : extracted into a function")
print("  callers added       : two, in ordinary feature work")
print("  did either author know : no; nobody knew yet")
print("  call sites at review time : " + str(call_sites_when_the_revert_was_written))
print("")
print("the clean apply")
print("  what a conflict would indicate : the original lines moved")
print("  what a clean apply indicates   : they did not")
print("  what either indicates about copies elsewhere : nothing")
print("  files the original commit touched : the one it touched")
print("  files the copies live in : two others")
print("")
print("the suite")
print("  green before the defect : yes")
print("  green after the defect  : yes, which is why a human")
print("    found it and not a test")
print("  green after the revert  : yes")
print("  tests asserting the behaviour is gone : " + str(tests_asserting_the_defective_behaviour_is_gone))
print("  so the suite is measuring : that nothing else broke")
print("")
nc_call_sites_the_fix_removed = call_sites_when_the_revert_was_written
nc_call_sites_remaining = 0
nc_tests_asserting_the_defective_behaviour_is_gone = 1
print("null control - the finding is written as a failing test")
print("  defect found by review : the same reviewer, unchanged")
print("  tests asserting the behaviour is gone : " + str(nc_tests_asserting_the_defective_behaviour_is_gone))
print("  call sites the fix must reach : " + str(nc_call_sites_the_fix_removed))
print("  call sites remaining : " + str(nc_call_sites_remaining))
print("  the review did not get better; the finding stopped being")
print("  addressed to a commit and started being addressed to a")
print("  behaviour")
print("")
print("what a clean revert guarantees")
print("  the tree no longer contains that commit's changes :")
print("    exactly, and it was confirmed line by line")
print("  the defect is gone                                : not")
print("    addressed; a commit is a location in a history and a")
print("    defect is a behaviour that can be copied")
print("")
print("a fix aimed at a commit is aimed at the place the defect was")
print("introduced; between introduction and discovery the code")
print("moves, and only a test names the behaviour rather than the")
print("place")
print("")
print("The review did the hard part: a person read the diff, found a wrong comparison,")
print("was right, and " + str(defects_caught_in_review_this_year) + " defects were caught this way this year. The revert")
print("applied with " + str(conflicts_when_the_revert_applied) + " conflicts, was confirmed as the exact inverse of the original,")
print("and left " + str(failing_tests_after_the_revert) + " tests failing - and in the " + str(days_between_the_commit_and_the_review) + " days and " + str(commits_touching_the_file_in_between) + " commits since, the logic")
print("had been extracted, so " + str(call_sites_remaining) + " of " + str(call_sites_when_the_revert_was_written) + " call sites - " + str(remaining_per_myriad) + " per ten thousand -")
print("survived, with " + str(tests_asserting_the_defective_behaviour_is_gone) + " tests naming the behaviour.")

defects caught in review this year : 340
conflicts when the revert applied  : 0
second reviewers confirming the inverse : 1
failing tests after the revert     : 0

days between the commit and the review : 26
commits touching the file in between   : 14

call sites when the defect was introduced : 1
call sites when the revert was written    : 3
  removed by the revert                   : 1
  remaining                               : 2
  share remaining                         : 6666 per ten thousand
tests asserting the behaviour is gone     : 0

the review
  found by : a person reading the diff
  the finding : a comparison that was wrong
  was it right : yes, and the author agreed within the hour
  defects caught this way this year : 340
  verdict : CAUGHT

  a reviewer who reads carefully enough to find a wrong
  comparison is the most valuable thing in this list

the revert
  applied with conflicts : 0
  confirmed as the exact inverse of the original diff : by
    1 second reviewer
  suite after it : 0 failing
  what it restored : the lines that commit changed
  verdict : INVERTED

  every one of those checks passed, and each of them is a
  statement about the diff

the file between the commit and the review
  commits touching it : 14
  the logic was       : extracted into a function
  callers added       : two, in ordinary feature work
  did either author know : no; nobody knew yet
  call sites at review time : 3

the clean apply
  what a conflict would indicate : the original lines moved
  what a clean apply indicates   : they did not
  what either indicates about copies elsewhere : nothing
  files the original commit touched : the one it touched
  files the copies live in : two others

the suite
  green before the defect : yes
  green after the defect  : yes, which is why a human
    found it and not a test
  green after the revert  : yes
  tests asserting the behaviour is gone : 0
  so the suite is measuring : that nothing else broke

null control - the finding is written as a failing test
  defect found by review : the same reviewer, unchanged
  tests asserting the behaviour is gone : 1
  call sites the fix must reach : 3
  call sites remaining : 0
  the review did not get better; the finding stopped being
  addressed to a commit and started being addressed to a
  behaviour

what a clean revert guarantees
  the tree no longer contains that commit's changes :
    exactly, and it was confirmed line by line
  the defect is gone                                : not
    addressed; a commit is a location in a history and a
    defect is a behaviour that can be copied

a fix aimed at a commit is aimed at the place the defect was
introduced; between introduction and discovery the code
moves, and only a test names the behaviour rather than the
place

The review did the hard part: a person read the diff, found a wrong comparison,
was right, and 340 defects were caught this way this year. The revert
applied with 0 conflicts, was confirmed as the exact inverse of the original,
and left 0 tests failing - and in the 26 days and 14 commits since, the logic
had been extracted, so 2 of 3 call sites - 6666 per ten thousand -
survived, with 0 tests naming the behaviour.

Trace event types