# Example 877 — The run stopped before it had seen them all `the_run_stopped_before_it_had_seen_them_all.eml` - A fuzzing run draws random inputs that each trigger one of ten equally likely error categories, and after ten draws it reports which categories can occur. Every draw was genuine and every category seen was logged. How many draws it takes to see all ten is computed below. ## EML ```eml # Self-authored for the EML case corpus (no external origin). A fuzzing run draws # random inputs that each trigger one of ten equally likely error categories, and # after ten draws it reports which categories can occur. Every draw was genuine # and every category seen was logged. How many draws it takes to see all ten is # computed below. # # The run is careful. Every input is a genuine uniform draw from the engine's # generator; every triggered category is logged exactly; the run makes as many # draws as there are categories; and the intent is exactly 'which error # categories can this code produce'. # # Seeing all of n equally likely categories takes about n times the n-th harmonic # number of draws on average - twenty-nine for ten - so a run of ten draws is # expected to leave about a third of them unseen. 10 => categories 10 => draws_run 6 => distinct_categories_seen 1000 + 500 + 333 + 250 + 200 + 166 + 142 + 125 + 111 + 100 => harmonic_ten_per_mille int(categories * harmonic_ten_per_mille / 1000) => expected_draws_to_see_all categories - distinct_categories_seen => categories_declared_unreachable int(draws_run * 10000 / expected_draws_to_see_all) => share_of_needed_draws_made_per_myriad "error categories, equally likely : " + str(categories) ^0 "draws the run made : " + str(draws_run) ^0 "distinct categories seen : " + str(distinct_categories_seen) ^0 "categories declared unreachable : " + str(categories_declared_unreachable) ^0 "" ^0 "harmonic number H(10), per mille : " + str(harmonic_ten_per_mille) ^0 "expected draws to see all ten : " + str(expected_draws_to_see_all) ^0 "share of needed draws made : " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand" ^0 "" ^0 # ---- what the run verified ---- "the fuzzing run" ^0 " each input : a genuine uniform draw from the generator" ^0 " each category : logged exactly when triggered" ^0 " draws made : " + str(draws_run) + ", one per category" ^0 " intent : which error categories can this code produce" ^0 " draws that were not random : 0" ^0 " verdict : SIX CATEGORIES OBSERVED IN TEN DRAWS" ^0 "" ^0 " logging every triggered category from genuine random" ^0 " inputs is the part done right here, and it is why the six" ^0 " seen are certainly reachable" ^0 "" ^0 # ---- how many draws it takes to see them all ---- "the coupon collector" ^0 " the first draw : a new category, certainly" ^0 " the last new category : found at one chance in ten per" ^0 " draw, about ten draws of waiting" ^0 " the total, summed over all ten : ten times H(10), about" ^0 " " + str(expected_draws_to_see_all) + " draws" ^0 " what ten draws are expected to reach : about six or" ^0 " seven distinct categories" ^0 " so four unseen after ten : the expected outcome, not a" ^0 " finding" ^0 "" ^0 # ---- what the caller got ---- "the report" ^0 " categories declared unreachable : " + str(categories_declared_unreachable) ^0 " categories that actually exist : " + str(categories) ^0 " draws made vs draws needed : " + str(draws_run) + " of about " + str(expected_draws_to_see_all) ^0 " is any seen category wrong : no; all six are real" ^0 " is 'not seen in ten draws' the same as 'cannot occur' :" ^0 " no; the run made " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand of the" ^0 " draws needed to expect a full set" ^0 "" ^0 # ---- null control ---- # The same run, continued to the coupon-collector expectation and beyond (or # until the last new category is followed by a long silence). 6 => nc_distinct_after_ten_draws 10 => nc_distinct_after_the_expected_draws 4 => nc_categories_the_longer_run_recovers "null control - run to the expected number of draws" ^0 " distinct after ten draws : " + str(nc_distinct_after_ten_draws) ^0 " distinct after about " + str(expected_draws_to_see_all) + " draws : " + str(nc_distinct_after_the_expected_draws) ^0 " categories the longer run recovers : " + str(nc_categories_the_longer_run_recovers) ^0 " no code and no generator changed; the run stopped being" ^0 " cut off at the count of categories" ^0 "" ^0 # ---- the rule ---- "what a ten-draw fuzzing run guarantees" ^0 " the categories it saw are reachable : exactly, genuine" ^0 " draws, exact logging" ^0 " the categories it did not see are unreachable : not" ^0 " addressed; collecting all ten takes about " + str(expected_draws_to_see_all) + " draws on" ^0 " average, so ten draws expect to miss about a third -" ^0 " the " + str(categories_declared_unreachable) + " declared unreachable are simply not yet drawn" ^0 "" ^0 "each new kind is easy to find while most are still unfound and hard once few" ^0 "remain; collecting them all costs n times the harmonic sum, not n, and a run" ^0 "sized to the count of kinds stops while the rare tail is still expected" ^0 "" ^0 "It logs every category from genuine random draws - the six seen are real. But" ^0 "collecting all ten equally likely kinds takes about " + str(expected_draws_to_see_all) + " draws on average, and" ^0 "the run made " + str(draws_run) + ", " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand of that; the " + str(categories_declared_unreachable) + " categories it declared" ^0 "unreachable were the expected leftover, until the run was continued." ^0 ``` ## Python (deterministic transpilation) ```python categories = 10 draws_run = 10 distinct_categories_seen = 6 harmonic_ten_per_mille = 1000 + 500 + 333 + 250 + 200 + 166 + 142 + 125 + 111 + 100 expected_draws_to_see_all = int(categories * harmonic_ten_per_mille / 1000) categories_declared_unreachable = categories - distinct_categories_seen share_of_needed_draws_made_per_myriad = int(draws_run * 10000 / expected_draws_to_see_all) print("error categories, equally likely : " + str(categories)) print("draws the run made : " + str(draws_run)) print("distinct categories seen : " + str(distinct_categories_seen)) print("categories declared unreachable : " + str(categories_declared_unreachable)) print("") print("harmonic number H(10), per mille : " + str(harmonic_ten_per_mille)) print("expected draws to see all ten : " + str(expected_draws_to_see_all)) print("share of needed draws made : " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand") print("") print("the fuzzing run") print(" each input : a genuine uniform draw from the generator") print(" each category : logged exactly when triggered") print(" draws made : " + str(draws_run) + ", one per category") print(" intent : which error categories can this code produce") print(" draws that were not random : 0") print(" verdict : SIX CATEGORIES OBSERVED IN TEN DRAWS") print("") print(" logging every triggered category from genuine random") print(" inputs is the part done right here, and it is why the six") print(" seen are certainly reachable") print("") print("the coupon collector") print(" the first draw : a new category, certainly") print(" the last new category : found at one chance in ten per") print(" draw, about ten draws of waiting") print(" the total, summed over all ten : ten times H(10), about") print(" " + str(expected_draws_to_see_all) + " draws") print(" what ten draws are expected to reach : about six or") print(" seven distinct categories") print(" so four unseen after ten : the expected outcome, not a") print(" finding") print("") print("the report") print(" categories declared unreachable : " + str(categories_declared_unreachable)) print(" categories that actually exist : " + str(categories)) print(" draws made vs draws needed : " + str(draws_run) + " of about " + str(expected_draws_to_see_all)) print(" is any seen category wrong : no; all six are real") print(" is 'not seen in ten draws' the same as 'cannot occur' :") print(" no; the run made " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand of the") print(" draws needed to expect a full set") print("") nc_distinct_after_ten_draws = 6 nc_distinct_after_the_expected_draws = 10 nc_categories_the_longer_run_recovers = 4 print("null control - run to the expected number of draws") print(" distinct after ten draws : " + str(nc_distinct_after_ten_draws)) print(" distinct after about " + str(expected_draws_to_see_all) + " draws : " + str(nc_distinct_after_the_expected_draws)) print(" categories the longer run recovers : " + str(nc_categories_the_longer_run_recovers)) print(" no code and no generator changed; the run stopped being") print(" cut off at the count of categories") print("") print("what a ten-draw fuzzing run guarantees") print(" the categories it saw are reachable : exactly, genuine") print(" draws, exact logging") print(" the categories it did not see are unreachable : not") print(" addressed; collecting all ten takes about " + str(expected_draws_to_see_all) + " draws on") print(" average, so ten draws expect to miss about a third -") print(" the " + str(categories_declared_unreachable) + " declared unreachable are simply not yet drawn") print("") print("each new kind is easy to find while most are still unfound and hard once few") print("remain; collecting them all costs n times the harmonic sum, not n, and a run") print("sized to the count of kinds stops while the rare tail is still expected") print("") print("It logs every category from genuine random draws - the six seen are real. But") print("collecting all ten equally likely kinds takes about " + str(expected_draws_to_see_all) + " draws on average, and") print("the run made " + str(draws_run) + ", " + str(share_of_needed_draws_made_per_myriad) + " per ten thousand of that; the " + str(categories_declared_unreachable) + " categories it declared") print("unreachable were the expected leftover, until the run was continued.") ``` ## stdout (executed) ```text error categories, equally likely : 10 draws the run made : 10 distinct categories seen : 6 categories declared unreachable : 4 harmonic number H(10), per mille : 2927 expected draws to see all ten : 29 share of needed draws made : 3448 per ten thousand the fuzzing run each input : a genuine uniform draw from the generator each category : logged exactly when triggered draws made : 10, one per category intent : which error categories can this code produce draws that were not random : 0 verdict : SIX CATEGORIES OBSERVED IN TEN DRAWS logging every triggered category from genuine random inputs is the part done right here, and it is why the six seen are certainly reachable the coupon collector the first draw : a new category, certainly the last new category : found at one chance in ten per draw, about ten draws of waiting the total, summed over all ten : ten times H(10), about 29 draws what ten draws are expected to reach : about six or seven distinct categories so four unseen after ten : the expected outcome, not a finding the report categories declared unreachable : 4 categories that actually exist : 10 draws made vs draws needed : 10 of about 29 is any seen category wrong : no; all six are real is 'not seen in ten draws' the same as 'cannot occur' : no; the run made 3448 per ten thousand of the draws needed to expect a full set null control - run to the expected number of draws distinct after ten draws : 6 distinct after about 29 draws : 10 categories the longer run recovers : 4 no code and no generator changed; the run stopped being cut off at the count of categories what a ten-draw fuzzing run guarantees the categories it saw are reachable : exactly, genuine draws, exact logging the categories it did not see are unreachable : not addressed; collecting all ten takes about 29 draws on average, so ten draws expect to miss about a third - the 4 declared unreachable are simply not yet drawn each new kind is easy to find while most are still unfound and hard once few remain; collecting them all costs n times the harmonic sum, not n, and a run sized to the count of kinds stops while the rare tail is still expected It logs every category from genuine random draws - the six seen are real. But collecting all ten equally likely kinds takes about 29 draws on average, and the run made 10, 3448 per ten thousand of that; the 4 categories it declared unreachable were the expected leftover, until the run was continued. ``` ## Round-trip `ok: true` — round-trip fixpoint reached (python1 == python2) ## Trace event types eml:run:start · eml:assign · eml:output · eml:run:done