{
  "id": "P144",
  "slug": "customer-merge",
  "key": "P144-customer-merge",
  "title": "Customer merge",
  "summary": "Duplicate customers merged by a matcher that compares stored strings and one that normalises when it compares - merges the first job's own writes manufacture - and the orders that still name the rows merged away.",
  "entry": "main.eml",
  "ui": "terminal",
  "readme": "# P144 - Customer merge\n\nDuplicate customers merged by two jobs, merge by merge, and the orders that\nstill name the rows merged away.\n\n- `main.eml` - the menu, the rows and each job's groups, one merge at a\n  time or until done, the orders joined three ways, and starting over\n- `dedupe.eml` - the customers and orders, how two records match, the next\n  match, a merge, whether original rows matched, and the orders a join\n  still finds\n\nHow each part works:\n\n- Eight rows, typed by different people: Ana's phone as `555-0101` and\n  `5550101`, Bo's as `555 0202` and `5550202`, Cy's as `555-0303` and\n  `5550303`; some rows have only an email.\n- A merge keeps the newer row and writes the phone in its canonical form,\n  digits only. The raw job's matcher compares the stored strings; the\n  compare-time job normalises the phone when it compares.\n- As in the corpus case `merge-invents-a-match`, the raw job finds Ana's\n  rows 1 and 3 by email, and the merge writes `5550101` - which now equals\n  row 2 as typed. The next merge rests on that value: no original row of\n  one side matches one of the other. Bo's rows go the same way. Cy has no\n  email to start the chain, so the raw job never joins rows 7 and 8: 4\n  merges, 2 of them manufactured. The compare-time job makes 5, none\n  manufactured.\n- As in the corpus case `survivor-id-still-referenced`, the orders keep\n  naming the row merged away. With the merged rows hard-deleted, a join\n  finds 6 of the 12 orders after the raw job and 5 after the other. A\n  redirect from each merged row to its survivor finds them all only when it\n  is followed to the end: row 4 was merged into 5, and 5 into 6, so one\n  redirect lands on a row that is gone.\n\nWhat is checked: menu choices 1 to 6.\n\nSessions: `sessions/basic.in` gives:\n- the rows and groups; one merge each and the orders;\n- three merges more and the orders, with one redirect falling short;\n- merging until done, the orders, and the groups at the end.\n\n`sessions/bad-input.in` gives:\n- menu choices 0, x and 7;\n- merging until done, then a next merge with nothing left to match; the\n  orders; starting over, and the orders with nothing merged.\n\nBuilt on the verified corpus cases `merge-invents-a-match` (a merge that\nwrites a canonical value, and the match it manufactures for a matcher that\ncompares stored strings) and `survivor-id-still-referenced` (the loser's id\nstill named by the orders, under a hard delete and a redirect).\n",
  "modules": [
    {
      "name": "main.eml",
      "eml": "# P144 customer merge: the same duplicate customers merged by two jobs - a\n# matcher that compares the stored strings and one that normalises when it\n# compares - merge by merge, and the orders joined to their customers after\n# a hard delete, one redirect, or every redirect.\nimport dedupe\n\ndef plain(s):\n    0 => i\n    len(s) => j\n    while i < j and s[i] == \" \":\n        i + 1 => i\n    while j > i and s[j - 1] == \" \":\n        j - 1 => j\n    return s[i:j]\n\ndef pad(s, n):\n    s => out\n    while len(out) < n:\n        out + \" \" => out\n    return out\n\ndef lpad(s, n):\n    s => out\n    while len(out) < n:\n        \" \" + out => out\n    return out\n\ndef q(s):\n    return \"'\" + s + \"'\"\n\ndef ids(xs):\n    \"\" => out\n    for x in xs:\n        if out == \"\":\n            str(x) => out\n        else:\n            out + \", \" + str(x) => out\n    return out\n\ndef times(n, one, many):\n    if n == 1:\n        return \"1 \" + one\n    return str(n) + \" \" + many\n\ndef label(normalise):\n    if normalise:\n        return \"Compare-time matcher\"\n    return \"Raw matcher\"\n\ndef show_groups(normalise, gs):\n    if normalise:\n        (label(normalise) + \" (normalises before comparing), groups:\") ^0\n    else:\n        (label(normalise) + \" (compares the stored strings), groups:\") ^0\n    for g in gs:\n        (\"  \" + pad(str(g[0]) + \": rows \" + ids(g[1]), 20) + pad(\"phone \" + q(g[2]), 18) + \"email \" + q(g[3])) ^0\n\ndef show(r, n):\n    \"The rows, as typed:\" ^0\n    for c in dedupe.customers():\n        (\"  \" + str(c[0]) + \"  \" + pad(c[1], 11) + pad(q(c[2]), 12) + q(c[3])) ^0\n    show_groups(False, r[0])\n    show_groups(True, n[0])\n\ndef step(normalise, w):\n    # one merge in world w = [groups, redirects, merges, manufactured];\n    # returns [the world, True when there was a match]\n    w[0] => gs\n    dedupe.next_pair(normalise, gs) => p\n    if len(p) == 0:\n        (label(normalise) + \": no two records match any more.\") ^0\n        return [w, False]\n    p[0] => i\n    p[1] => j\n    dedupe.evidence(normalise, gs[i], gs[j]) => ev\n    if ev == \"email\":\n        \"email \" + gs[i][3] => said\n    elif normalise and gs[i][2] != gs[j][2]:\n        \"phone \" + q(gs[i][2]) + \" and \" + q(gs[j][2]) + \", both \" + dedupe.digits(gs[i][2]) => said\n    else:\n        \"phone \" + q(gs[i][2]) + \" in both\" => said\n    dedupe.loser_of(gs, i, j) => lo\n    dedupe.survivor_of(gs, i, j) => su\n    w[3] => made\n    if dedupe.rows_match(normalise, gs[i][1], gs[j][1]):\n        \"an original row of each side matches: real.\" => tail\n    else:\n        \"no original row of one side matches one of the other: manufactured by a value a merge wrote.\" => tail\n        made + 1 => made\n    (label(normalise) + \": merged \" + str(lo) + \" into \" + str(su) + \" - \" + said + \"; \" + tail) ^0\n    {} => red\n    for k in w[1]:\n        w[1][k] => red[k]\n    su => red[lo]\n    return [[dedupe.merge(gs, i, j), red, w[2] + 1, made], True]\n\ndef fresh():\n    return [dedupe.start(), {}, 0, 0]\n\ndef done(normalise, w):\n    return len(dedupe.next_pair(normalise, w[0])) == 0\n\ndef group_of(gs, id):\n    for g in gs:\n        for x in g[1]:\n            if x == id:\n                return g[0]\n    return 0\n\ndef reports(r, n):\n    len(dedupe.orders()) => total\n    (\"Orders, \" + str(total) + \" in all, joined to their customer after the merges so far:\") ^0\n    \"                                           raw   compare-time\" ^0\n    (\"  \" + pad(\"counted without a join\", 38) + lpad(str(total), 5) + lpad(str(total), 15)) ^0\n    (\"  \" + pad(\"joined, merged rows hard-deleted\", 38) + lpad(str(dedupe.joined(r[0], r[1], 0)), 5) + lpad(str(dedupe.joined(n[0], n[1], 0)), 15)) ^0\n    (\"  \" + pad(\"joined, following one redirect\", 38) + lpad(str(dedupe.joined(r[0], r[1], 1)), 5) + lpad(str(dedupe.joined(n[0], n[1], 1)), 15)) ^0\n    (\"  \" + pad(\"joined, following every redirect\", 38) + lpad(str(dedupe.joined(r[0], r[1], 9)), 5) + lpad(str(dedupe.joined(n[0], n[1], 9)), 15)) ^0\n    if dedupe.joined(r[0], r[1], 0) < total or dedupe.joined(n[0], n[1], 0) < total:\n        \"Hard-deleted: an order still names the row merged away, and the join drops it without an error.\" ^0\n    if dedupe.joined(r[0], r[1], 1) < total or dedupe.joined(n[0], n[1], 1) < total:\n        \"One redirect: an order whose row was merged into one that was merged again lands on a row that is gone.\" ^0\n    (\"Raw matcher: \" + times(r[2], \"merge\", \"merges\") + \" so far, \" + str(r[3]) + \" of them manufactured.\") ^0\n    (\"Compare-time matcher: \" + times(n[2], \"merge\", \"merges\") + \" so far, \" + str(n[3]) + \" of them manufactured.\") ^0\n    if done(False, r) and done(True, n):\n        \"\" => apart\n        for a in dedupe.customers():\n            for b in dedupe.customers():\n                if a[0] < b[0] and group_of(n[0], a[0]) == group_of(n[0], b[0]) and group_of(r[0], a[0]) != group_of(r[0], b[0]):\n                    if apart == \"\":\n                        str(a[0]) + \" and \" + str(b[0]) => apart\n                    else:\n                        apart + \"; \" + str(a[0]) + \" and \" + str(b[0]) => apart\n        if apart == \"\":\n            \"Both jobs are done and ended with the same groups.\" ^0\n        else:\n            (\"Both jobs are done. Rows the compare-time matcher put together and the raw job never did: \" + apart + \".\") ^0\n    else:\n        \"Merge until done to see where the two jobs end.\" ^0\n\nfresh() => r\nfresh() => n\n\n\"== Customer merge ==\" ^0\n\"Duplicate customers merged by two matchers, and the orders that still name the rows merged away.\" ^0\nTrue => running\nwhile running:\n    \"\" ^0\n    \"1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\" ^0\n    plain(input(\"choice> \")) => choice\n    if choice == \"1\":\n        show(r, n)\n    elif choice == \"2\":\n        step(False, r)[0] => r\n        step(True, n)[0] => n\n    elif choice == \"3\":\n        True => more\n        while more:\n            False => more\n            if not done(False, r):\n                step(False, r)[0] => r\n                True => more\n            if not done(True, n):\n                step(True, n)[0] => n\n                True => more\n        \"Both jobs are done.\" ^0\n    elif choice == \"4\":\n        reports(r, n)\n    elif choice == \"5\":\n        fresh() => r\n        fresh() => n\n        \"Back to the rows as typed, nothing merged.\" ^0\n    elif choice == \"6\":\n        False => running\n    else:\n        \"Pick a number from 1 to 6.\" ^0\n\"Bye.\" ^0\n",
      "python": "import dedupe\n\ndef plain(s):\n    i = 0\n    j = len(s)\n    while i < j and s[i] == \" \":\n        i = i + 1\n    while j > i and s[j - 1] == \" \":\n        j = j - 1\n    return s[i:j]\n\ndef pad(s, n):\n    out = s\n    while len(out) < n:\n        out = out + \" \"\n    return out\n\ndef lpad(s, n):\n    out = s\n    while len(out) < n:\n        out = \" \" + out\n    return out\n\ndef q(s):\n    return \"'\" + s + \"'\"\n\ndef ids(xs):\n    out = \"\"\n    for x in xs:\n        if out == \"\":\n            out = str(x)\n        else:\n            out = out + \", \" + str(x)\n    return out\n\ndef times(n, one, many):\n    if n == 1:\n        return \"1 \" + one\n    return str(n) + \" \" + many\n\ndef label(normalise):\n    if normalise:\n        return \"Compare-time matcher\"\n    return \"Raw matcher\"\n\ndef show_groups(normalise, gs):\n    if normalise:\n        print(label(normalise) + \" (normalises before comparing), groups:\")\n    else:\n        print(label(normalise) + \" (compares the stored strings), groups:\")\n    for g in gs:\n        print(\"  \" + pad(str(g[0]) + \": rows \" + ids(g[1]), 20) + pad(\"phone \" + q(g[2]), 18) + \"email \" + q(g[3]))\n\ndef show(r, n):\n    print(\"The rows, as typed:\")\n    for c in dedupe.customers():\n        print(\"  \" + str(c[0]) + \"  \" + pad(c[1], 11) + pad(q(c[2]), 12) + q(c[3]))\n    show_groups(False, r[0])\n    show_groups(True, n[0])\n\ndef step(normalise, w):\n    gs = w[0]\n    p = dedupe.next_pair(normalise, gs)\n    if len(p) == 0:\n        print(label(normalise) + \": no two records match any more.\")\n        return [w, False]\n    i = p[0]\n    j = p[1]\n    ev = dedupe.evidence(normalise, gs[i], gs[j])\n    if ev == \"email\":\n        said = \"email \" + gs[i][3]\n    elif normalise and gs[i][2] != gs[j][2]:\n        said = \"phone \" + q(gs[i][2]) + \" and \" + q(gs[j][2]) + \", both \" + dedupe.digits(gs[i][2])\n    else:\n        said = \"phone \" + q(gs[i][2]) + \" in both\"\n    lo = dedupe.loser_of(gs, i, j)\n    su = dedupe.survivor_of(gs, i, j)\n    made = w[3]\n    if dedupe.rows_match(normalise, gs[i][1], gs[j][1]):\n        tail = \"an original row of each side matches: real.\"\n    else:\n        tail = \"no original row of one side matches one of the other: manufactured by a value a merge wrote.\"\n        made = made + 1\n    print(label(normalise) + \": merged \" + str(lo) + \" into \" + str(su) + \" - \" + said + \"; \" + tail)\n    red = {}\n    for k in w[1]:\n        red[k] = w[1][k]\n    red[lo] = su\n    return [[dedupe.merge(gs, i, j), red, w[2] + 1, made], True]\n\ndef fresh():\n    return [dedupe.start(), {}, 0, 0]\n\ndef done(normalise, w):\n    return len(dedupe.next_pair(normalise, w[0])) == 0\n\ndef group_of(gs, id):\n    for g in gs:\n        for x in g[1]:\n            if x == id:\n                return g[0]\n    return 0\n\ndef reports(r, n):\n    total = len(dedupe.orders())\n    print(\"Orders, \" + str(total) + \" in all, joined to their customer after the merges so far:\")\n    print(\"                                           raw   compare-time\")\n    print(\"  \" + pad(\"counted without a join\", 38) + lpad(str(total), 5) + lpad(str(total), 15))\n    print(\"  \" + pad(\"joined, merged rows hard-deleted\", 38) + lpad(str(dedupe.joined(r[0], r[1], 0)), 5) + lpad(str(dedupe.joined(n[0], n[1], 0)), 15))\n    print(\"  \" + pad(\"joined, following one redirect\", 38) + lpad(str(dedupe.joined(r[0], r[1], 1)), 5) + lpad(str(dedupe.joined(n[0], n[1], 1)), 15))\n    print(\"  \" + pad(\"joined, following every redirect\", 38) + lpad(str(dedupe.joined(r[0], r[1], 9)), 5) + lpad(str(dedupe.joined(n[0], n[1], 9)), 15))\n    if dedupe.joined(r[0], r[1], 0) < total or dedupe.joined(n[0], n[1], 0) < total:\n        print(\"Hard-deleted: an order still names the row merged away, and the join drops it without an error.\")\n    if dedupe.joined(r[0], r[1], 1) < total or dedupe.joined(n[0], n[1], 1) < total:\n        print(\"One redirect: an order whose row was merged into one that was merged again lands on a row that is gone.\")\n    print(\"Raw matcher: \" + times(r[2], \"merge\", \"merges\") + \" so far, \" + str(r[3]) + \" of them manufactured.\")\n    print(\"Compare-time matcher: \" + times(n[2], \"merge\", \"merges\") + \" so far, \" + str(n[3]) + \" of them manufactured.\")\n    if done(False, r) and done(True, n):\n        apart = \"\"\n        for a in dedupe.customers():\n            for b in dedupe.customers():\n                if a[0] < b[0] and group_of(n[0], a[0]) == group_of(n[0], b[0]) and group_of(r[0], a[0]) != group_of(r[0], b[0]):\n                    if apart == \"\":\n                        apart = str(a[0]) + \" and \" + str(b[0])\n                    else:\n                        apart = apart + \"; \" + str(a[0]) + \" and \" + str(b[0])\n        if apart == \"\":\n            print(\"Both jobs are done and ended with the same groups.\")\n        else:\n            print(\"Both jobs are done. Rows the compare-time matcher put together and the raw job never did: \" + apart + \".\")\n    else:\n        print(\"Merge until done to see where the two jobs end.\")\n\nr = fresh()\nn = fresh()\nprint(\"== Customer merge ==\")\nprint(\"Duplicate customers merged by two matchers, and the orders that still name the rows merged away.\")\nrunning = True\nwhile running:\n    print(\"\")\n    print(\"1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\")\n    choice = plain(input(\"choice> \"))\n    if choice == \"1\":\n        show(r, n)\n    elif choice == \"2\":\n        r = step(False, r)[0]\n        n = step(True, n)[0]\n    elif choice == \"3\":\n        more = True\n        while more:\n            more = False\n            if not done(False, r):\n                r = step(False, r)[0]\n                more = True\n            if not done(True, n):\n                n = step(True, n)[0]\n                more = True\n        print(\"Both jobs are done.\")\n    elif choice == \"4\":\n        reports(r, n)\n    elif choice == \"5\":\n        r = fresh()\n        n = fresh()\n        print(\"Back to the rows as typed, nothing merged.\")\n    elif choice == \"6\":\n        running = False\n    else:\n        print(\"Pick a number from 1 to 6.\")\nprint(\"Bye.\")\n"
    },
    {
      "name": "dedupe.eml",
      "eml": "# P144 customer merge - duplicate customers found by a matcher and merged,\n# and the orders that still name the rows merged away. A group is\n# [survivor, [member rows], stored phone, stored email]: the record the\n# matcher compares. The newer row survives, and a merge writes the phone in\n# its canonical form, digits only. As in the corpus case\n# merge-invents-a-match, a matcher that compares the stored strings can then\n# find a match no two original rows have: the merge made the evidence. The\n# matcher that normalises when it compares cannot. As in the corpus case\n# survivor-id-still-referenced, the orders keep naming the loser: a hard\n# delete drops them from every join, a redirect keeps them only if it is\n# followed to the end.\n\ndef customers():\n    # [id, name, phone as typed, email]\n    return [[1, \"Ana Silva\", \"555-0101\", \"ana@x\"], [2, \"Ana Silva\", \"5550101\", \"\"], [3, \"Ana S.\", \"\", \"ana@x\"], [4, \"Bo Chen\", \"555 0202\", \"bo@x\"], [5, \"Bo Chen\", \"\", \"bo@x\"], [6, \"B. Chen\", \"5550202\", \"\"], [7, \"Cy Diaz\", \"555-0303\", \"cy@x\"], [8, \"Cy Diaz\", \"5550303\", \"\"]]\n\ndef orders():\n    # [order number, customer id]\n    return [[101, 1], [102, 2], [103, 3], [104, 3], [105, 4], [106, 5], [107, 6], [108, 6], [109, 7], [110, 8], [111, 1], [112, 5]]\n\ndef row(id):\n    for c in customers():\n        if c[0] == id:\n            return c\n    return []\n\ndef digits(s):\n    \"\" => out\n    for c in s:\n        if c >= \"0\" and c <= \"9\":\n            out + c => out\n    return out\n\ndef start():\n    [] => gs\n    for c in customers():\n        gs + [[c[0], [c[0]], c[2], c[3]]] => gs\n    return gs\n\ndef evidence(normalise, a, b):\n    # how two stored records match: \"phone\", \"email\" or \"\"\n    if normalise:\n        if digits(a[2]) != \"\" and digits(a[2]) == digits(b[2]):\n            return \"phone\"\n    elif a[2] != \"\" and a[2] == b[2]:\n        return \"phone\"\n    if a[3] != \"\" and a[3] == b[3]:\n        return \"email\"\n    return \"\"\n\ndef rows_match(normalise, ids1, ids2):\n    # does an original row of one side match an original row of the other\n    for x in ids1:\n        for y in ids2:\n            row(x) => a\n            row(y) => b\n            if evidence(normalise, [a[0], [], a[2], a[3]], [b[0], [], b[2], b[3]]) != \"\":\n                return True\n    return False\n\ndef next_pair(normalise, gs):\n    # positions [i, j] of the first two groups that match, or []\n    0 => i\n    while i < len(gs):\n        i + 1 => j\n        while j < len(gs):\n            if evidence(normalise, gs[i], gs[j]) != \"\":\n                return [i, j]\n            j + 1 => j\n        i + 1 => i\n    return []\n\ndef merge(gs, i, j):\n    # the groups with i and j made one; the newer survivor stays and the\n    # merge writes the phone digits only\n    gs[i] => a\n    gs[j] => b\n    if a[0] > b[0]:\n        a => keep\n        b => lose\n    else:\n        b => keep\n        a => lose\n    keep[2] => phone\n    if phone == \"\":\n        lose[2] => phone\n    keep[3] => email\n    if email == \"\":\n        lose[3] => email\n    [keep[0], a[1] + b[1], digits(phone), email] => g\n    [] => out\n    0 => k\n    for x in gs:\n        if k == i:\n            out + [g] => out\n        elif k != j:\n            out + [x] => out\n        k + 1 => k\n    return out\n\ndef loser_of(gs, i, j):\n    if gs[i][0] > gs[j][0]:\n        return gs[j][0]\n    return gs[i][0]\n\ndef survivor_of(gs, i, j):\n    if gs[i][0] > gs[j][0]:\n        return gs[i][0]\n    return gs[j][0]\n\ndef is_survivor(gs, id):\n    for g in gs:\n        if g[0] == id:\n            return True\n    return False\n\ndef joined(gs, redirect, hops):\n    # orders whose customer is found: hops 0 is a hard delete, 1 follows one\n    # redirect, more follows them that many times\n    0 => n\n    for o in orders():\n        o[1] => id\n        0 => h\n        while not is_survivor(gs, id) and h < hops and id in redirect:\n            redirect[id] => id\n            h + 1 => h\n        if is_survivor(gs, id):\n            n + 1 => n\n    return n\n",
      "python": "def customers():\n    return [[1, \"Ana Silva\", \"555-0101\", \"ana@x\"], [2, \"Ana Silva\", \"5550101\", \"\"], [3, \"Ana S.\", \"\", \"ana@x\"], [4, \"Bo Chen\", \"555 0202\", \"bo@x\"], [5, \"Bo Chen\", \"\", \"bo@x\"], [6, \"B. Chen\", \"5550202\", \"\"], [7, \"Cy Diaz\", \"555-0303\", \"cy@x\"], [8, \"Cy Diaz\", \"5550303\", \"\"]]\n\ndef orders():\n    return [[101, 1], [102, 2], [103, 3], [104, 3], [105, 4], [106, 5], [107, 6], [108, 6], [109, 7], [110, 8], [111, 1], [112, 5]]\n\ndef row(id):\n    for c in customers():\n        if c[0] == id:\n            return c\n    return []\n\ndef digits(s):\n    out = \"\"\n    for c in s:\n        if c >= \"0\" and c <= \"9\":\n            out = out + c\n    return out\n\ndef start():\n    gs = []\n    for c in customers():\n        gs = gs + [[c[0], [c[0]], c[2], c[3]]]\n    return gs\n\ndef evidence(normalise, a, b):\n    if normalise:\n        if digits(a[2]) != \"\" and digits(a[2]) == digits(b[2]):\n            return \"phone\"\n    elif a[2] != \"\" and a[2] == b[2]:\n        return \"phone\"\n    if a[3] != \"\" and a[3] == b[3]:\n        return \"email\"\n    return \"\"\n\ndef rows_match(normalise, ids1, ids2):\n    for x in ids1:\n        for y in ids2:\n            a = row(x)\n            b = row(y)\n            if evidence(normalise, [a[0], [], a[2], a[3]], [b[0], [], b[2], b[3]]) != \"\":\n                return True\n    return False\n\ndef next_pair(normalise, gs):\n    i = 0\n    while i < len(gs):\n        j = i + 1\n        while j < len(gs):\n            if evidence(normalise, gs[i], gs[j]) != \"\":\n                return [i, j]\n            j = j + 1\n        i = i + 1\n    return []\n\ndef merge(gs, i, j):\n    a = gs[i]\n    b = gs[j]\n    if a[0] > b[0]:\n        keep = a\n        lose = b\n    else:\n        keep = b\n        lose = a\n    phone = keep[2]\n    if phone == \"\":\n        phone = lose[2]\n    email = keep[3]\n    if email == \"\":\n        email = lose[3]\n    g = [keep[0], a[1] + b[1], digits(phone), email]\n    out = []\n    k = 0\n    for x in gs:\n        if k == i:\n            out = out + [g]\n        elif k != j:\n            out = out + [x]\n        k = k + 1\n    return out\n\ndef loser_of(gs, i, j):\n    if gs[i][0] > gs[j][0]:\n        return gs[j][0]\n    return gs[i][0]\n\ndef survivor_of(gs, i, j):\n    if gs[i][0] > gs[j][0]:\n        return gs[i][0]\n    return gs[j][0]\n\ndef is_survivor(gs, id):\n    for g in gs:\n        if g[0] == id:\n            return True\n    return False\n\ndef joined(gs, redirect, hops):\n    n = 0\n    for o in orders():\n        id = o[1]\n        h = 0\n        while not is_survivor(gs, id) and h < hops and id in redirect:\n            id = redirect[id]\n            h = h + 1\n        if is_survivor(gs, id):\n            n = n + 1\n    return n\n"
    }
  ],
  "sessions": [
    {
      "name": "bad-input",
      "input": "0\nx\n7\n3\n2\n4\n5\n4\n6\n",
      "screen": "== Customer merge ==\nDuplicate customers merged by two matchers, and the orders that still name the rows merged away.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 0\nPick a number from 1 to 6.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> x\nPick a number from 1 to 6.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 7\nPick a number from 1 to 6.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 3\nRaw matcher: merged 1 into 3 - email ana@x; an original row of each side matches: real.\nCompare-time matcher: merged 1 into 2 - phone '555-0101' and '5550101', both 5550101; an original row of each side matches: real.\nRaw matcher: merged 2 into 3 - phone '5550101' in both; no original row of one side matches one of the other: manufactured by a value a merge wrote.\nCompare-time matcher: merged 2 into 3 - email ana@x; an original row of each side matches: real.\nRaw matcher: merged 4 into 5 - email bo@x; an original row of each side matches: real.\nCompare-time matcher: merged 4 into 5 - email bo@x; an original row of each side matches: real.\nRaw matcher: merged 5 into 6 - phone '5550202' in both; no original row of one side matches one of the other: manufactured by a value a merge wrote.\nCompare-time matcher: merged 5 into 6 - phone '5550202' in both; an original row of each side matches: real.\nCompare-time matcher: merged 7 into 8 - phone '555-0303' and '5550303', both 5550303; an original row of each side matches: real.\nBoth jobs are done.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 2\nRaw matcher: no two records match any more.\nCompare-time matcher: no two records match any more.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 4\nOrders, 12 in all, joined to their customer after the merges so far:\n                                           raw   compare-time\n  counted without a join                   12             12\n  joined, merged rows hard-deleted          6              5\n  joined, following one redirect           11              9\n  joined, following every redirect         12             12\nHard-deleted: an order still names the row merged away, and the join drops it without an error.\nOne redirect: an order whose row was merged into one that was merged again lands on a row that is gone.\nRaw matcher: 4 merges so far, 2 of them manufactured.\nCompare-time matcher: 5 merges so far, 0 of them manufactured.\nBoth jobs are done. Rows the compare-time matcher put together and the raw job never did: 7 and 8.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 5\nBack to the rows as typed, nothing merged.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 4\nOrders, 12 in all, joined to their customer after the merges so far:\n                                           raw   compare-time\n  counted without a join                   12             12\n  joined, merged rows hard-deleted         12             12\n  joined, following one redirect           12             12\n  joined, following every redirect         12             12\nRaw matcher: 0 merges so far, 0 of them manufactured.\nCompare-time matcher: 0 merges so far, 0 of them manufactured.\nMerge until done to see where the two jobs end.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 6\nBye.\n",
      "interpreter": "equal"
    },
    {
      "name": "basic",
      "input": "1\n2\n4\n2\n2\n2\n4\n3\n4\n1\n6\n",
      "screen": "== Customer merge ==\nDuplicate customers merged by two matchers, and the orders that still name the rows merged away.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 1\nThe rows, as typed:\n  1  Ana Silva  '555-0101'  'ana@x'\n  2  Ana Silva  '5550101'   ''\n  3  Ana S.     ''          'ana@x'\n  4  Bo Chen    '555 0202'  'bo@x'\n  5  Bo Chen    ''          'bo@x'\n  6  B. Chen    '5550202'   ''\n  7  Cy Diaz    '555-0303'  'cy@x'\n  8  Cy Diaz    '5550303'   ''\nRaw matcher (compares the stored strings), groups:\n  1: rows 1           phone '555-0101'  email 'ana@x'\n  2: rows 2           phone '5550101'   email ''\n  3: rows 3           phone ''          email 'ana@x'\n  4: rows 4           phone '555 0202'  email 'bo@x'\n  5: rows 5           phone ''          email 'bo@x'\n  6: rows 6           phone '5550202'   email ''\n  7: rows 7           phone '555-0303'  email 'cy@x'\n  8: rows 8           phone '5550303'   email ''\nCompare-time matcher (normalises before comparing), groups:\n  1: rows 1           phone '555-0101'  email 'ana@x'\n  2: rows 2           phone '5550101'   email ''\n  3: rows 3           phone ''          email 'ana@x'\n  4: rows 4           phone '555 0202'  email 'bo@x'\n  5: rows 5           phone ''          email 'bo@x'\n  6: rows 6           phone '5550202'   email ''\n  7: rows 7           phone '555-0303'  email 'cy@x'\n  8: rows 8           phone '5550303'   email ''\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 2\nRaw matcher: merged 1 into 3 - email ana@x; an original row of each side matches: real.\nCompare-time matcher: merged 1 into 2 - phone '555-0101' and '5550101', both 5550101; an original row of each side matches: real.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 4\nOrders, 12 in all, joined to their customer after the merges so far:\n                                           raw   compare-time\n  counted without a join                   12             12\n  joined, merged rows hard-deleted         10             10\n  joined, following one redirect           12             12\n  joined, following every redirect         12             12\nHard-deleted: an order still names the row merged away, and the join drops it without an error.\nRaw matcher: 1 merge so far, 0 of them manufactured.\nCompare-time matcher: 1 merge so far, 0 of them manufactured.\nMerge until done to see where the two jobs end.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 2\nRaw matcher: merged 2 into 3 - phone '5550101' in both; no original row of one side matches one of the other: manufactured by a value a merge wrote.\nCompare-time matcher: merged 2 into 3 - email ana@x; an original row of each side matches: real.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 2\nRaw matcher: merged 4 into 5 - email bo@x; an original row of each side matches: real.\nCompare-time matcher: merged 4 into 5 - email bo@x; an original row of each side matches: real.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 2\nRaw matcher: merged 5 into 6 - phone '5550202' in both; no original row of one side matches one of the other: manufactured by a value a merge wrote.\nCompare-time matcher: merged 5 into 6 - phone '5550202' in both; an original row of each side matches: real.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 4\nOrders, 12 in all, joined to their customer after the merges so far:\n                                           raw   compare-time\n  counted without a join                   12             12\n  joined, merged rows hard-deleted          6              6\n  joined, following one redirect           11              9\n  joined, following every redirect         12             12\nHard-deleted: an order still names the row merged away, and the join drops it without an error.\nOne redirect: an order whose row was merged into one that was merged again lands on a row that is gone.\nRaw matcher: 4 merges so far, 2 of them manufactured.\nCompare-time matcher: 4 merges so far, 0 of them manufactured.\nMerge until done to see where the two jobs end.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 3\nCompare-time matcher: merged 7 into 8 - phone '555-0303' and '5550303', both 5550303; an original row of each side matches: real.\nBoth jobs are done.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 4\nOrders, 12 in all, joined to their customer after the merges so far:\n                                           raw   compare-time\n  counted without a join                   12             12\n  joined, merged rows hard-deleted          6              5\n  joined, following one redirect           11              9\n  joined, following every redirect         12             12\nHard-deleted: an order still names the row merged away, and the join drops it without an error.\nOne redirect: an order whose row was merged into one that was merged again lands on a row that is gone.\nRaw matcher: 4 merges so far, 2 of them manufactured.\nCompare-time matcher: 5 merges so far, 0 of them manufactured.\nBoth jobs are done. Rows the compare-time matcher put together and the raw job never did: 7 and 8.\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 1\nThe rows, as typed:\n  1  Ana Silva  '555-0101'  'ana@x'\n  2  Ana Silva  '5550101'   ''\n  3  Ana S.     ''          'ana@x'\n  4  Bo Chen    '555 0202'  'bo@x'\n  5  Bo Chen    ''          'bo@x'\n  6  B. Chen    '5550202'   ''\n  7  Cy Diaz    '555-0303'  'cy@x'\n  8  Cy Diaz    '5550303'   ''\nRaw matcher (compares the stored strings), groups:\n  3: rows 1, 3, 2     phone '5550101'   email 'ana@x'\n  6: rows 4, 5, 6     phone '5550202'   email 'bo@x'\n  7: rows 7           phone '555-0303'  email 'cy@x'\n  8: rows 8           phone '5550303'   email ''\nCompare-time matcher (normalises before comparing), groups:\n  3: rows 1, 2, 3     phone '5550101'   email 'ana@x'\n  6: rows 4, 5, 6     phone '5550202'   email 'bo@x'\n  8: rows 7, 8        phone '5550303'   email 'cy@x'\n\n1) rows and groups  2) next merge  3) merge until done  4) orders  5) start over  6) quit\nchoice> 6\nBye.\n",
      "interpreter": "equal"
    }
  ],
  "builtOn": [
    {
      "slug": "merge-invents-a-match",
      "caseId": "301-merge-invents-a-match",
      "title": "Merge invents a match — one sweep wrote a value no record held, the next believed it"
    },
    {
      "slug": "survivor-id-still-referenced",
      "caseId": "305-survivor-id-still-referenced",
      "title": "Survivor id still referenced — the rows vanished from one report and stayed in the other"
    }
  ],
  "updated": "2026-10-18"
}