{
  "id": "P157",
  "slug": "fraud-alerts",
  "key": "P157-fraud-alerts",
  "title": "Fraud alerts",
  "summary": "A detector right 99% of the time over a million payments a month: accuracy that stays put while precision follows the fraud rate, the rate at which alerts are right as often as not, and a second check on the alerts.",
  "entry": "main.eml",
  "ui": "terminal",
  "readme": "# P157 - Fraud alerts\n\nA detector right 99% of the time, run over a million card payments a\nmonth, and how often one of its alerts is fraud.\n\n- `main.eml` - the menu, a month through the detector, the fraud rate and\n  the detector set by hand, every fraud rate side by side, and the alerts\n  checked a second time\n- `fraud.eml` - the month, the four counts, percentages in tenths rounded\n  half up, precision and accuracy, the fraud rate at which alerts are right\n  as often as not, and a second check\n\nHow each part works:\n\n- The detector catches 99.0% of frauds and flags 1.0% of honest payments.\n  With 200 frauds in a million payments it is right about 99.0% of them,\n  and of its 10196 alerts 198 are fraud: precision 1.9%.\n- As in the corpus case `base-rate-and-precision`, accuracy is about the\n  detector and precision is about the world: at every fraud rate from 1 to\n  100000 a month accuracy stays 99.0%, while precision runs from 0.0% to\n  91.7%. Alerts are fraud as often as not only from 10000 frauds a month -\n  found from the formula and then checked past the rounding.\n- The alerts are a population of their own, with a fraud rate equal to\n  the first precision. Checked again by an independent detector just as\n  good, 196 of the 295 left are fraud: 66.4%.\n- With the detector set by hand: catching nothing makes no alert right at\n  any rate; flagging nothing honest makes every alert right from the first\n  caught fraud.\n\nWhat is checked: menu choices 1 to 6; frauds a month from 0 to 1000000;\nthe catch rate and the false alarm rate from 0 to 1000 tenths of a\npercent. An empty answer cancels. Every count is rounded down.\n\nSessions: `sessions/basic.in` gives:\n- a month, every fraud rate, and the alerts checked again;\n- 20000 frauds a month, a month and the alerts checked again;\n- the detector at 99.9% and 0.1%, a month and every fraud rate.\n\n`sessions/bad-input.in` gives:\n- menu choices 0 and x;\n- fraud rates: a cancel, 1000001 and x;\n- detectors: a cancel and 1001, then 990 with a cancel and x; a detector\n  that catches nothing, every rate and a month; one that catches 0.5% and\n  flags nothing honest, every rate and the alerts checked again.\n\nBuilt on the verified corpus case `base-rate-and-precision` (a fixed test\nswept across base rates, precision against accuracy, and the rate at which\na positive is more likely right than wrong).\n",
  "modules": [
    {
      "name": "main.eml",
      "eml": "# P157 fraud alerts: a month of payments through the detector, the fraud\n# rate and the detector set by hand, every fraud rate side by side, and the\n# alerts checked a second time.\nimport fraud\n\ndef plain(s):\n    0 => i\n    len(s) => j\n    while i < j and s[i] == \" \":\n        i + 1 => i\n    while j > i and s[j - 1] == \" \":\n        j - 1 => j\n    return s[i:j]\n\ndef pad(s, n):\n    s => out\n    while len(out) < n:\n        out + \" \" => out\n    return out\n\ndef lpad(s, n):\n    s => out\n    while len(out) < n:\n        \" \" + out => out\n    return out\n\ndef pct(t):\n    t % 10 => d\n    return str(int((t - d) / 10)) + \".\" + str(d) + \"%\"\n\ndef number(prompt, high):\n    # a whole number from 0 to high, -1 for a cancel, -2 when not valid\n    plain(input(prompt)) => t\n    if t == \"\":\n        return 0 - 1\n    if len(t) > 7:\n        return 0 - 2\n    for c in t:\n        if c < \"0\" or c > \"9\":\n            return 0 - 2\n    if int(t) > high:\n        return 0 - 2\n    return int(t)\n\ndef even_line(catch, alarm):\n    fraud.even(catch, alarm) => b\n    if b < 0:\n        \"A detector that catches nothing makes no alert right, at any fraud rate.\" ^0\n    else:\n        if b == 1:\n            \"An alert is fraud at least as often as not from the first fraud of the month.\" ^0\n        else:\n            (\"An alert is fraud at least as often as not only from \" + str(b) + \" frauds a month.\") ^0\n\ndef show_month(f, catch, alarm):\n    fraud.outcomes(f, catch, alarm) => r\n    (\"A month of \" + str(fraud.month()) + \" payments, \" + str(f) + \" of them fraud; the detector catches \" + pct(catch) + \" of frauds and flags \" + pct(alarm) + \" of honest payments:\") ^0\n    (\"  \" + pad(\"\", 9) + lpad(\"flagged\", 10) + lpad(\"passed\", 10)) ^0\n    (\"  \" + pad(\"fraud\", 9) + lpad(str(r[0]), 10) + lpad(str(r[1]), 10)) ^0\n    (\"  \" + pad(\"honest\", 9) + lpad(str(r[2]), 10) + lpad(str(r[3]), 10)) ^0\n    (\"Accuracy \" + pct(fraud.accuracy(r)) + \": the detector is right about that share of payments.\") ^0\n    r[0] + r[2] => alerts\n    if alerts == 0:\n        \"No alerts this month.\" ^0\n    else:\n        if alerts == 1:\n            \"Of the 1 alert, \" + str(r[0]) + \" is fraud\" => head\n        else:\n            \"Of the \" + str(alerts) + \" alerts, \" + str(r[0]) + \" are fraud\" => head\n        (head + \": precision \" + pct(fraud.precision(r)) + \". Act on an alert and you are wrong \" + pct(1000 - fraud.precision(r)) + \" of the time.\") ^0\n    even_line(catch, alarm)\n\ndef sweep(catch, alarm):\n    (\"The same detector, \" + pct(catch) + \" caught and \" + pct(alarm) + \" false alarms, at every fraud rate:\") ^0\n    (\"  \" + pad(\"frauds a month\", 16) + pad(\"accuracy\", 10) + pad(\"alerts\", 9) + \"precision\") ^0\n    1001 => alo\n    0 - 1 => ahi\n    1001 => plo\n    0 - 1 => phi\n    for f in [1, 10, 100, 1000, 10000, 100000]:\n        fraud.outcomes(f, catch, alarm) => r\n        fraud.accuracy(r) => a\n        fraud.precision(r) => p\n        if r[0] + r[2] == 0:\n            (\"  \" + pad(str(f), 16) + pad(pct(a), 10) + pad(\"0\", 9) + \"-\") ^0\n        else:\n            (\"  \" + pad(str(f), 16) + pad(pct(a), 10) + pad(str(r[0] + r[2]), 9) + pct(p)) ^0\n            if p < plo:\n                p => plo\n            if p > phi:\n                p => phi\n        if a < alo:\n            a => alo\n        if a > ahi:\n            a => ahi\n    if alo == ahi:\n        \"Accuracy is \" + pct(alo) + \" at every rate\" => acc\n    else:\n        \"Accuracy runs from \" + pct(alo) + \" to \" + pct(ahi) => acc\n    if phi < 0:\n        (acc + \"; at none of these rates is there an alert.\") ^0\n    elif plo == phi:\n        (acc + \"; precision is \" + pct(plo) + \" wherever there are alerts.\") ^0\n    else:\n        (acc + \"; precision runs from \" + pct(plo) + \" to \" + pct(phi) + \", following the fraud rate, which the detector does not know.\") ^0\n    even_line(catch, alarm)\n\ndef again(f, catch, alarm):\n    fraud.outcomes(f, catch, alarm) => r\n    fraud.second(r, catch, alarm) => s\n    r[0] + r[2] => alerts\n    if alerts == 0:\n        \"No alerts this month, so nothing to check again.\" ^0\n        return\n    if alerts == 1:\n        \"The 1 alert checked again by an independent detector just as good:\" ^0\n    else:\n        (\"The \" + str(alerts) + \" alerts checked again by an independent detector just as good:\") ^0\n    (\"  first check:   \" + str(r[0]) + \" fraud of \" + str(alerts) + \", precision \" + pct(fraud.precision(r))) ^0\n    if s[0] + s[1] == 0:\n        \"  second check:  no alert left\" ^0\n    else:\n        (\"  second check:  \" + str(s[0]) + \" fraud of \" + str(s[0] + s[1]) + \", precision \" + pct(fraud.tenths(s[0], s[0] + s[1]))) ^0\n        \"The alerts are a population of their own, with a fraud rate equal to the first precision; the second check starts from there.\" ^0\n    (\"  frauds missed by the two together: \" + str(f - s[0])) ^0\n\n200 => f\n990 => catch\n10 => alarm\n\n\"== Fraud alerts ==\" ^0\n\"A detector right 99% of the time, a million payments a month, and how often an alert is fraud.\" ^0\nTrue => running\nwhile running:\n    \"\" ^0\n    \"1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\" ^0\n    plain(input(\"choice> \")) => choice\n    if choice == \"1\":\n        show_month(f, catch, alarm)\n    elif choice == \"2\":\n        number(\"frauds in a month of 1000000 payments> \", fraud.month()) => n\n        if n == 0 - 1:\n            \"Cancelled.\" ^0\n        elif n < 0:\n            \"Type a whole number from 0 to 1000000.\" ^0\n        else:\n            n => f\n            (\"Fraud set to \" + str(f) + \" a month.\") ^0\n    elif choice == \"3\":\n        number(\"frauds caught, in tenths of a percent (0 to 1000)> \", 1000) => a\n        if a == 0 - 1:\n            \"Cancelled.\" ^0\n        elif a < 0:\n            \"Type a whole number from 0 to 1000.\" ^0\n        else:\n            number(\"honest payments flagged, in tenths of a percent (0 to 1000)> \", 1000) => b\n            if b == 0 - 1:\n                \"Cancelled.\" ^0\n            elif b < 0:\n                \"Type a whole number from 0 to 1000.\" ^0\n            else:\n                a => catch\n                b => alarm\n                (\"The detector now catches \" + pct(catch) + \" of frauds and flags \" + pct(alarm) + \" of honest payments.\") ^0\n    elif choice == \"4\":\n        sweep(catch, alarm)\n    elif choice == \"5\":\n        again(f, catch, alarm)\n    elif choice == \"6\":\n        False => running\n    else:\n        \"Pick a number from 1 to 6.\" ^0\n\"Bye.\" ^0\n",
      "python": "import fraud\n\ndef plain(s):\n    i = 0\n    j = len(s)\n    while i < j and s[i] == \" \":\n        i = i + 1\n    while j > i and s[j - 1] == \" \":\n        j = j - 1\n    return s[i:j]\n\ndef pad(s, n):\n    out = s\n    while len(out) < n:\n        out = out + \" \"\n    return out\n\ndef lpad(s, n):\n    out = s\n    while len(out) < n:\n        out = \" \" + out\n    return out\n\ndef pct(t):\n    d = t % 10\n    return str(int((t - d) / 10)) + \".\" + str(d) + \"%\"\n\ndef number(prompt, high):\n    t = plain(input(prompt))\n    if t == \"\":\n        return 0 - 1\n    if len(t) > 7:\n        return 0 - 2\n    for c in t:\n        if c < \"0\" or c > \"9\":\n            return 0 - 2\n    if int(t) > high:\n        return 0 - 2\n    return int(t)\n\ndef even_line(catch, alarm):\n    b = fraud.even(catch, alarm)\n    if b < 0:\n        print(\"A detector that catches nothing makes no alert right, at any fraud rate.\")\n    elif b == 1:\n        print(\"An alert is fraud at least as often as not from the first fraud of the month.\")\n    else:\n        print(\"An alert is fraud at least as often as not only from \" + str(b) + \" frauds a month.\")\n\ndef show_month(f, catch, alarm):\n    r = fraud.outcomes(f, catch, alarm)\n    print(\"A month of \" + str(fraud.month()) + \" payments, \" + str(f) + \" of them fraud; the detector catches \" + pct(catch) + \" of frauds and flags \" + pct(alarm) + \" of honest payments:\")\n    print(\"  \" + pad(\"\", 9) + lpad(\"flagged\", 10) + lpad(\"passed\", 10))\n    print(\"  \" + pad(\"fraud\", 9) + lpad(str(r[0]), 10) + lpad(str(r[1]), 10))\n    print(\"  \" + pad(\"honest\", 9) + lpad(str(r[2]), 10) + lpad(str(r[3]), 10))\n    print(\"Accuracy \" + pct(fraud.accuracy(r)) + \": the detector is right about that share of payments.\")\n    alerts = r[0] + r[2]\n    if alerts == 0:\n        print(\"No alerts this month.\")\n    else:\n        if alerts == 1:\n            head = \"Of the 1 alert, \" + str(r[0]) + \" is fraud\"\n        else:\n            head = \"Of the \" + str(alerts) + \" alerts, \" + str(r[0]) + \" are fraud\"\n        print(head + \": precision \" + pct(fraud.precision(r)) + \". Act on an alert and you are wrong \" + pct(1000 - fraud.precision(r)) + \" of the time.\")\n    even_line(catch, alarm)\n\ndef sweep(catch, alarm):\n    print(\"The same detector, \" + pct(catch) + \" caught and \" + pct(alarm) + \" false alarms, at every fraud rate:\")\n    print(\"  \" + pad(\"frauds a month\", 16) + pad(\"accuracy\", 10) + pad(\"alerts\", 9) + \"precision\")\n    alo = 1001\n    ahi = 0 - 1\n    plo = 1001\n    phi = 0 - 1\n    for f in [1, 10, 100, 1000, 10000, 100000]:\n        r = fraud.outcomes(f, catch, alarm)\n        a = fraud.accuracy(r)\n        p = fraud.precision(r)\n        if r[0] + r[2] == 0:\n            print(\"  \" + pad(str(f), 16) + pad(pct(a), 10) + pad(\"0\", 9) + \"-\")\n        else:\n            print(\"  \" + pad(str(f), 16) + pad(pct(a), 10) + pad(str(r[0] + r[2]), 9) + pct(p))\n            if p < plo:\n                plo = p\n            if p > phi:\n                phi = p\n        if a < alo:\n            alo = a\n        if a > ahi:\n            ahi = a\n    if alo == ahi:\n        acc = \"Accuracy is \" + pct(alo) + \" at every rate\"\n    else:\n        acc = \"Accuracy runs from \" + pct(alo) + \" to \" + pct(ahi)\n    if phi < 0:\n        print(acc + \"; at none of these rates is there an alert.\")\n    elif plo == phi:\n        print(acc + \"; precision is \" + pct(plo) + \" wherever there are alerts.\")\n    else:\n        print(acc + \"; precision runs from \" + pct(plo) + \" to \" + pct(phi) + \", following the fraud rate, which the detector does not know.\")\n    even_line(catch, alarm)\n\ndef again(f, catch, alarm):\n    r = fraud.outcomes(f, catch, alarm)\n    s = fraud.second(r, catch, alarm)\n    alerts = r[0] + r[2]\n    if alerts == 0:\n        print(\"No alerts this month, so nothing to check again.\")\n        return\n    if alerts == 1:\n        print(\"The 1 alert checked again by an independent detector just as good:\")\n    else:\n        print(\"The \" + str(alerts) + \" alerts checked again by an independent detector just as good:\")\n    print(\"  first check:   \" + str(r[0]) + \" fraud of \" + str(alerts) + \", precision \" + pct(fraud.precision(r)))\n    if s[0] + s[1] == 0:\n        print(\"  second check:  no alert left\")\n    else:\n        print(\"  second check:  \" + str(s[0]) + \" fraud of \" + str(s[0] + s[1]) + \", precision \" + pct(fraud.tenths(s[0], s[0] + s[1])))\n        print(\"The alerts are a population of their own, with a fraud rate equal to the first precision; the second check starts from there.\")\n    print(\"  frauds missed by the two together: \" + str(f - s[0]))\n\nf = 200\ncatch = 990\nalarm = 10\nprint(\"== Fraud alerts ==\")\nprint(\"A detector right 99% of the time, a million payments a month, and how often an alert is fraud.\")\nrunning = True\nwhile running:\n    print(\"\")\n    print(\"1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\")\n    choice = plain(input(\"choice> \"))\n    if choice == \"1\":\n        show_month(f, catch, alarm)\n    elif choice == \"2\":\n        n = number(\"frauds in a month of 1000000 payments> \", fraud.month())\n        if n == 0 - 1:\n            print(\"Cancelled.\")\n        elif n < 0:\n            print(\"Type a whole number from 0 to 1000000.\")\n        else:\n            f = n\n            print(\"Fraud set to \" + str(f) + \" a month.\")\n    elif choice == \"3\":\n        a = number(\"frauds caught, in tenths of a percent (0 to 1000)> \", 1000)\n        if a == 0 - 1:\n            print(\"Cancelled.\")\n        elif a < 0:\n            print(\"Type a whole number from 0 to 1000.\")\n        else:\n            b = number(\"honest payments flagged, in tenths of a percent (0 to 1000)> \", 1000)\n            if b == 0 - 1:\n                print(\"Cancelled.\")\n            elif b < 0:\n                print(\"Type a whole number from 0 to 1000.\")\n            else:\n                catch = a\n                alarm = b\n                print(\"The detector now catches \" + pct(catch) + \" of frauds and flags \" + pct(alarm) + \" of honest payments.\")\n    elif choice == \"4\":\n        sweep(catch, alarm)\n    elif choice == \"5\":\n        again(f, catch, alarm)\n    elif choice == \"6\":\n        running = False\n    else:\n        print(\"Pick a number from 1 to 6.\")\nprint(\"Bye.\")\n"
    },
    {
      "name": "fraud.eml",
      "eml": "# P157 fraud alerts - a detector that is right 99% of the time, run over a\n# million card payments a month. As in the corpus case\n# base-rate-and-precision, accuracy is about the detector and precision -\n# how often an alert is fraud - is about the world: when fraud is rare, 1%\n# of the honest payments outnumbers 99% of the frauds, and most alerts are\n# wrong. Counts are whole payments over a fixed month; a rate is in tenths\n# of a percent.\n\ndef month():\n    return 1000000\n\ndef outcomes(fraud, catch, alarm):\n    # [frauds caught, frauds missed, honest flagged, honest passed] for a\n    # month with fraud frauds, catching catch and falsely flagging alarm\n    # tenths of a percent, each count rounded down\n    int(fraud * catch / 1000) => tp\n    month() - fraud => honest\n    int(honest * alarm / 1000) => fp\n    return [tp, fraud - tp, fp, honest - fp]\n\ndef tenths(n, d):\n    # n / d as a percentage in tenths, rounded half up; 0 when d is 0\n    if d == 0:\n        return 0\n    int((2000 * n + d) / (2 * d)) => q\n    while q * 2 * d > 2000 * n + d:\n        q - 1 => q\n    while (q + 1) * 2 * d <= 2000 * n + d:\n        q + 1 => q\n    return q\n\ndef precision(r):\n    return tenths(r[0], r[0] + r[2])\n\ndef accuracy(r):\n    return tenths(r[0] + r[3], month())\n\ndef right_half(b, catch, alarm):\n    # some fraud is caught, and the alerts are fraud at least as often as not\n    outcomes(b, catch, alarm) => r\n    return r[0] > 0 and r[0] >= r[2]\n\ndef even(catch, alarm):\n    # the fewest frauds a month at which an alert is fraud at least as\n    # often as not; the formula's answer, then nudged past the rounding\n    if catch == 0:\n        return 0 - 1\n    int(month() * alarm / (catch + alarm)) => b\n    if b < 1:\n        1 => b\n    while b > 1 and right_half(b - 1, catch, alarm):\n        b - 1 => b\n    while not right_half(b, catch, alarm):\n        b + 1 => b\n    return b\n\ndef second(r, catch, alarm):\n    # the alerts checked again by an independent detector just as good:\n    # [frauds still flagged, honest still flagged]\n    return [int(r[0] * catch / 1000), int(r[2] * alarm / 1000)]\n",
      "python": "def month():\n    return 1000000\n\ndef outcomes(fraud, catch, alarm):\n    tp = int(fraud * catch / 1000)\n    honest = month() - fraud\n    fp = int(honest * alarm / 1000)\n    return [tp, fraud - tp, fp, honest - fp]\n\ndef tenths(n, d):\n    if d == 0:\n        return 0\n    q = int((2000 * n + d) / (2 * d))\n    while q * 2 * d > 2000 * n + d:\n        q = q - 1\n    while (q + 1) * 2 * d <= 2000 * n + d:\n        q = q + 1\n    return q\n\ndef precision(r):\n    return tenths(r[0], r[0] + r[2])\n\ndef accuracy(r):\n    return tenths(r[0] + r[3], month())\n\ndef right_half(b, catch, alarm):\n    r = outcomes(b, catch, alarm)\n    return r[0] > 0 and r[0] >= r[2]\n\ndef even(catch, alarm):\n    if catch == 0:\n        return 0 - 1\n    b = int(month() * alarm / (catch + alarm))\n    if b < 1:\n        b = 1\n    while b > 1 and right_half(b - 1, catch, alarm):\n        b = b - 1\n    while not right_half(b, catch, alarm):\n        b = b + 1\n    return b\n\ndef second(r, catch, alarm):\n    return [int(r[0] * catch / 1000), int(r[2] * alarm / 1000)]\n"
    }
  ],
  "sessions": [
    {
      "name": "bad-input",
      "input": "0\nx\n2\n\n2\n1000001\n2\nx\n3\n\n3\n1001\n3\n990\n\n3\n990\nx\n3\n0\n10\n4\n1\n3\n5\n0\n4\n5\n6\n",
      "screen": "== Fraud alerts ==\nA detector right 99% of the time, a million payments a month, and how often an alert is fraud.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 0\nPick a number from 1 to 6.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> x\nPick a number from 1 to 6.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 2\nfrauds in a month of 1000000 payments> \nCancelled.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 2\nfrauds in a month of 1000000 payments> 1000001\nType a whole number from 0 to 1000000.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 2\nfrauds in a month of 1000000 payments> x\nType a whole number from 0 to 1000000.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> \nCancelled.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 1001\nType a whole number from 0 to 1000.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 990\nhonest payments flagged, in tenths of a percent (0 to 1000)> \nCancelled.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 990\nhonest payments flagged, in tenths of a percent (0 to 1000)> x\nType a whole number from 0 to 1000.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 0\nhonest payments flagged, in tenths of a percent (0 to 1000)> 10\nThe detector now catches 0.0% of frauds and flags 1.0% of honest payments.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 4\nThe same detector, 0.0% caught and 1.0% false alarms, at every fraud rate:\n  frauds a month  accuracy  alerts   precision\n  1               99.0%     9999     0.0%\n  10              99.0%     9999     0.0%\n  100             99.0%     9999     0.0%\n  1000            98.9%     9990     0.0%\n  10000           98.0%     9900     0.0%\n  100000          89.1%     9000     0.0%\nAccuracy runs from 89.1% to 99.0%; precision is 0.0% wherever there are alerts.\nA detector that catches nothing makes no alert right, at any fraud rate.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 1\nA month of 1000000 payments, 200 of them fraud; the detector catches 0.0% of frauds and flags 1.0% of honest payments:\n              flagged    passed\n  fraud             0       200\n  honest         9998    989802\nAccuracy 99.0%: the detector is right about that share of payments.\nOf the 9998 alerts, 0 are fraud: precision 0.0%. Act on an alert and you are wrong 100.0% of the time.\nA detector that catches nothing makes no alert right, at any fraud rate.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 5\nhonest payments flagged, in tenths of a percent (0 to 1000)> 0\nThe detector now catches 0.5% of frauds and flags 0.0% of honest payments.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 4\nThe same detector, 0.5% caught and 0.0% false alarms, at every fraud rate:\n  frauds a month  accuracy  alerts   precision\n  1               100.0%    0        -\n  10              100.0%    0        -\n  100             100.0%    0        -\n  1000            99.9%     5        100.0%\n  10000           99.0%     50       100.0%\n  100000          90.1%     500      100.0%\nAccuracy runs from 90.1% to 100.0%; precision is 100.0% wherever there are alerts.\nAn alert is fraud at least as often as not only from 200 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 5\nThe 1 alert checked again by an independent detector just as good:\n  first check:   1 fraud of 1, precision 100.0%\n  second check:  no alert left\n  frauds missed by the two together: 200\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 6\nBye.\n",
      "interpreter": "equal"
    },
    {
      "name": "basic",
      "input": "1\n4\n5\n2\n20000\n1\n5\n3\n999\n1\n1\n4\n6\n",
      "screen": "== Fraud alerts ==\nA detector right 99% of the time, a million payments a month, and how often an alert is fraud.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 1\nA month of 1000000 payments, 200 of them fraud; the detector catches 99.0% of frauds and flags 1.0% of honest payments:\n              flagged    passed\n  fraud           198         2\n  honest         9998    989802\nAccuracy 99.0%: the detector is right about that share of payments.\nOf the 10196 alerts, 198 are fraud: precision 1.9%. Act on an alert and you are wrong 98.1% of the time.\nAn alert is fraud at least as often as not only from 10000 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 4\nThe same detector, 99.0% caught and 1.0% false alarms, at every fraud rate:\n  frauds a month  accuracy  alerts   precision\n  1               99.0%     9999     0.0%\n  10              99.0%     10008    0.1%\n  100             99.0%     10098    1.0%\n  1000            99.0%     10980    9.0%\n  10000           99.0%     19800    50.0%\n  100000          99.0%     108000   91.7%\nAccuracy is 99.0% at every rate; precision runs from 0.0% to 91.7%, following the fraud rate, which the detector does not know.\nAn alert is fraud at least as often as not only from 10000 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 5\nThe 10196 alerts checked again by an independent detector just as good:\n  first check:   198 fraud of 10196, precision 1.9%\n  second check:  196 fraud of 295, precision 66.4%\nThe alerts are a population of their own, with a fraud rate equal to the first precision; the second check starts from there.\n  frauds missed by the two together: 4\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 2\nfrauds in a month of 1000000 payments> 20000\nFraud set to 20000 a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 1\nA month of 1000000 payments, 20000 of them fraud; the detector catches 99.0% of frauds and flags 1.0% of honest payments:\n              flagged    passed\n  fraud         19800       200\n  honest         9800    970200\nAccuracy 99.0%: the detector is right about that share of payments.\nOf the 29600 alerts, 19800 are fraud: precision 66.9%. Act on an alert and you are wrong 33.1% of the time.\nAn alert is fraud at least as often as not only from 10000 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 5\nThe 29600 alerts checked again by an independent detector just as good:\n  first check:   19800 fraud of 29600, precision 66.9%\n  second check:  19602 fraud of 19700, precision 99.5%\nThe alerts are a population of their own, with a fraud rate equal to the first precision; the second check starts from there.\n  frauds missed by the two together: 398\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 3\nfrauds caught, in tenths of a percent (0 to 1000)> 999\nhonest payments flagged, in tenths of a percent (0 to 1000)> 1\nThe detector now catches 99.9% of frauds and flags 0.1% of honest payments.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 1\nA month of 1000000 payments, 20000 of them fraud; the detector catches 99.9% of frauds and flags 0.1% of honest payments:\n              flagged    passed\n  fraud         19980        20\n  honest          980    979020\nAccuracy 99.9%: the detector is right about that share of payments.\nOf the 20960 alerts, 19980 are fraud: precision 95.3%. Act on an alert and you are wrong 4.7% of the time.\nAn alert is fraud at least as often as not only from 1000 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 4\nThe same detector, 99.9% caught and 0.1% false alarms, at every fraud rate:\n  frauds a month  accuracy  alerts   precision\n  1               99.9%     999      0.0%\n  10              99.9%     1008     0.9%\n  100             99.9%     1098     9.0%\n  1000            99.9%     1998     50.0%\n  10000           99.9%     10980    91.0%\n  100000          99.9%     100800   99.1%\nAccuracy is 99.9% at every rate; precision runs from 0.0% to 99.1%, following the fraud rate, which the detector does not know.\nAn alert is fraud at least as often as not only from 1000 frauds a month.\n\n1) a month  2) the fraud rate  3) the detector  4) every fraud rate  5) check the alerts again  6) quit\nchoice> 6\nBye.\n",
      "interpreter": "equal"
    }
  ],
  "builtOn": [
    {
      "slug": "base-rate-and-precision",
      "caseId": "261-base-rate-and-precision",
      "title": "Base rate and precision — 99% accurate, and most alerts are false"
    }
  ],
  "updated": "2026-10-18"
}