The sample was drawn with replacement

# Self-authored for the EML case corpus (no external origin). A quality review
# examines a random sample of one thousand records out of ten thousand, and every
# one of the thousand draws is a genuine uniform random pick. How many distinct
# records the thousand draws reach is computed below.
#
# The sampling is careful. It uses the engine's own random generator, not a
# hand-picked list; every draw is uniform over the whole population; exactly one
# thousand draws are made; and the intent is exactly 'one thousand distinct
# records reviewed'.
#
# Each draw is made with replacement - a record already drawn can be drawn again -
# so about forty-nine of the thousand draws land on a record already in the
# sample.

10000 => population
1000 => draws

draws - 1 => draws_minus_one
2 * population => twice_population
int(draws * draws_minus_one / twice_population) => expected_repeated_draws
draws - expected_repeated_draws => expected_distinct_records
int(expected_repeated_draws * 10000 / draws) => repeats_per_myriad_of_the_sample

"population                      : " + str(population) ^0
"draws                           : " + str(draws) ^0
"records the report says reviewed : " + str(draws) ^0
"" ^0
"expected draws that repeat      : " + str(expected_repeated_draws) ^0
"expected distinct records       : " + str(expected_distinct_records) ^0
"repeats in the sample           : " + str(repeats_per_myriad_of_the_sample) + " per ten thousand" ^0
"" ^0

# ---- what the sampling verified ----

"the random sample" ^0
"  generator : the engine's own, not a hand-picked list" ^0
"  each draw : uniform over the whole population" ^0
"  draws made : exactly " + str(draws) ^0
"  intent : one thousand distinct records reviewed" ^0
"  draws that were not random : 0" ^0
"  verdict : ONE THOUSAND UNIFORM RANDOM DRAWS" ^0
"" ^0
"  making every draw a genuine uniform pick is the part done" ^0
"  right here, and it is why no record was favoured and no" ^0
"  region of the population was skipped" ^0
"" ^0

# ---- what a draw with replacement can land on ----

"drawing with replacement" ^0
"  what each draw can pick : any of the " + str(population) + ", including" ^0
"    ones already drawn" ^0
"  chance two given draws coincide : one in " + str(population) ^0
"  pairs of draws among " + str(draws) + " : about half a million" ^0
"  so expected coinciding pairs : about " + str(expected_repeated_draws) ^0
"  what a repeat is in the report : a record counted twice" ^0
"    and a record never seen" ^0
"" ^0

# ---- what the caller got ----

"the review report" ^0
"  records reported reviewed : " + str(draws) ^0
"  distinct records actually reviewed : about " + str(expected_distinct_records) ^0
"  records reviewed twice : about " + str(expected_repeated_draws) ^0
"  is any draw non-random : no; every one is uniform" ^0
"  is a thousand draws a thousand records : no; " + str(repeats_per_myriad_of_the_sample) + " per" ^0
"    ten thousand of the draws revisit a record" ^0
"" ^0

# ---- null control ----

# The same review, sampling without replacement (shuffle the population and take
# the first thousand, or reject a repeated draw), so draws equal distinct records.
951 => nc_distinct_with_replacement
1000 => nc_distinct_without_replacement
49 => nc_records_the_change_recovers

"null control - sample without replacement" ^0
"  distinct records, with replacement : about " + str(nc_distinct_with_replacement) ^0
"  distinct records, without replacement : " + str(nc_distinct_without_replacement) ^0
"  records the change recovers : about " + str(nc_records_the_change_recovers) ^0
"  no record and no generator changed; a drawn record" ^0
"  stopped being eligible to be drawn again" ^0
"" ^0

# ---- the rule ----

"what a thousand uniform draws guarantee" ^0
"  a thousand uniform random picks were made : exactly, the" ^0
"    engine's own generator, every draw" ^0
"  a thousand distinct records were reviewed : not" ^0
"    addressed; the draws are with replacement, so about " + str(expected_repeated_draws) ^0
"    of them land on a record already drawn and the distinct" ^0
"    count is about " + str(expected_distinct_records) ^0
"" ^0

"a count of draws is a count of events, and a count of records is a count of" ^0
"things; with replacement the two part ways as soon as the sample is a" ^0
"non-trivial fraction of the population, and the report added up the events" ^0
"" ^0

"It makes a thousand genuine uniform draws with the engine's own generator - no" ^0
"record is favoured. But the draws are with replacement, so about " + str(expected_repeated_draws) + " of them" ^0
"revisit a record and about " + str(expected_distinct_records) + " distinct records were reviewed, " + str(repeats_per_myriad_of_the_sample) + " per ten" ^0
"thousand of the sample being repeats, until the sampling was without replacement." ^0

population = 10000
draws = 1000
draws_minus_one = draws - 1
twice_population = 2 * population
expected_repeated_draws = int(draws * draws_minus_one / twice_population)
expected_distinct_records = draws - expected_repeated_draws
repeats_per_myriad_of_the_sample = int(expected_repeated_draws * 10000 / draws)
print("population                      : " + str(population))
print("draws                           : " + str(draws))
print("records the report says reviewed : " + str(draws))
print("")
print("expected draws that repeat      : " + str(expected_repeated_draws))
print("expected distinct records       : " + str(expected_distinct_records))
print("repeats in the sample           : " + str(repeats_per_myriad_of_the_sample) + " per ten thousand")
print("")
print("the random sample")
print("  generator : the engine's own, not a hand-picked list")
print("  each draw : uniform over the whole population")
print("  draws made : exactly " + str(draws))
print("  intent : one thousand distinct records reviewed")
print("  draws that were not random : 0")
print("  verdict : ONE THOUSAND UNIFORM RANDOM DRAWS")
print("")
print("  making every draw a genuine uniform pick is the part done")
print("  right here, and it is why no record was favoured and no")
print("  region of the population was skipped")
print("")
print("drawing with replacement")
print("  what each draw can pick : any of the " + str(population) + ", including")
print("    ones already drawn")
print("  chance two given draws coincide : one in " + str(population))
print("  pairs of draws among " + str(draws) + " : about half a million")
print("  so expected coinciding pairs : about " + str(expected_repeated_draws))
print("  what a repeat is in the report : a record counted twice")
print("    and a record never seen")
print("")
print("the review report")
print("  records reported reviewed : " + str(draws))
print("  distinct records actually reviewed : about " + str(expected_distinct_records))
print("  records reviewed twice : about " + str(expected_repeated_draws))
print("  is any draw non-random : no; every one is uniform")
print("  is a thousand draws a thousand records : no; " + str(repeats_per_myriad_of_the_sample) + " per")
print("    ten thousand of the draws revisit a record")
print("")
nc_distinct_with_replacement = 951
nc_distinct_without_replacement = 1000
nc_records_the_change_recovers = 49
print("null control - sample without replacement")
print("  distinct records, with replacement : about " + str(nc_distinct_with_replacement))
print("  distinct records, without replacement : " + str(nc_distinct_without_replacement))
print("  records the change recovers : about " + str(nc_records_the_change_recovers))
print("  no record and no generator changed; a drawn record")
print("  stopped being eligible to be drawn again")
print("")
print("what a thousand uniform draws guarantee")
print("  a thousand uniform random picks were made : exactly, the")
print("    engine's own generator, every draw")
print("  a thousand distinct records were reviewed : not")
print("    addressed; the draws are with replacement, so about " + str(expected_repeated_draws))
print("    of them land on a record already drawn and the distinct")
print("    count is about " + str(expected_distinct_records))
print("")
print("a count of draws is a count of events, and a count of records is a count of")
print("things; with replacement the two part ways as soon as the sample is a")
print("non-trivial fraction of the population, and the report added up the events")
print("")
print("It makes a thousand genuine uniform draws with the engine's own generator - no")
print("record is favoured. But the draws are with replacement, so about " + str(expected_repeated_draws) + " of them")
print("revisit a record and about " + str(expected_distinct_records) + " distinct records were reviewed, " + str(repeats_per_myriad_of_the_sample) + " per ten")
print("thousand of the sample being repeats, until the sampling was without replacement.")

population                      : 10000
draws                           : 1000
records the report says reviewed : 1000

expected draws that repeat      : 49
expected distinct records       : 951
repeats in the sample           : 490 per ten thousand

the random sample
  generator : the engine's own, not a hand-picked list
  each draw : uniform over the whole population
  draws made : exactly 1000
  intent : one thousand distinct records reviewed
  draws that were not random : 0
  verdict : ONE THOUSAND UNIFORM RANDOM DRAWS

  making every draw a genuine uniform pick is the part done
  right here, and it is why no record was favoured and no
  region of the population was skipped

drawing with replacement
  what each draw can pick : any of the 10000, including
    ones already drawn
  chance two given draws coincide : one in 10000
  pairs of draws among 1000 : about half a million
  so expected coinciding pairs : about 49
  what a repeat is in the report : a record counted twice
    and a record never seen

the review report
  records reported reviewed : 1000
  distinct records actually reviewed : about 951
  records reviewed twice : about 49
  is any draw non-random : no; every one is uniform
  is a thousand draws a thousand records : no; 490 per
    ten thousand of the draws revisit a record

null control - sample without replacement
  distinct records, with replacement : about 951
  distinct records, without replacement : 1000
  records the change recovers : about 49
  no record and no generator changed; a drawn record
  stopped being eligible to be drawn again

what a thousand uniform draws guarantee
  a thousand uniform random picks were made : exactly, the
    engine's own generator, every draw
  a thousand distinct records were reviewed : not
    addressed; the draws are with replacement, so about 49
    of them land on a record already drawn and the distinct
    count is about 951

a count of draws is a count of events, and a count of records is a count of
things; with replacement the two part ways as soon as the sample is a
non-trivial fraction of the population, and the report added up the events

It makes a thousand genuine uniform draws with the engine's own generator - no
record is favoured. But the draws are with replacement, so about 49 of them
revisit a record and about 951 distinct records were reviewed, 490 per ten
thousand of the sample being repeats, until the sampling was without replacement.

Trace event types