Signup funnel
Sign-up rates by segment with events over humans, clamped and matched, and the sign-ups credited to onboarding tips set against the ones they caused as more people see them.
Every screen below was recorded under CPython. When this page was built, the EML interpreter replayed each session from the same input and printed the same bytes.
About
Sign-up rates by traffic segment, worked out three ways, and the sign-ups credited to an onboarding feature set against the ones it caused.
main.eml- the menu, the segments, the rates, the tips at one rollout, and at every rolloutfunnel.eml- the segments, percentages in tenths rounded half up, the three rates, the buckets of visitors, and a rollout's counts
How each part works:
- Four segments - search, ads, social, email - each with human visitors and sign-ups, and bot and test visitors and sign-ups. As in the corpus case
rate-numerator-outgrows-denominator, the sign-ups come off the event table, which has every one, and the visitors off a view of humans. - Events over humans: ads reaches 135.0%, the one rate anyone notices, and the other three are high without looking wrong - social 35.0% against 10.0%. Clamped at 100%, ads reads 100.0%: all four are still wrong and nothing is left to notice. Humans over humans is the rate.
- As in the corpus case
usage-reported-as-effect, the tips are shown to the most engaged buckets first, and "sign-ups by users who saw the tips" counts everyone who passed through them. Shown to every bucket that count is 184, all the sign-ups; the tips caused 27. Down the buckets the credited count falls while the caused count never does: the most engaged sign up anyway.
What is checked: menu choices 1 to 5; a number of buckets from 0 to 10. An empty answer cancels.
Sessions: sessions/basic.in gives: - the segments and the rates; - the tips shown to 5 buckets and to all 10; every rollout.
sessions/bad-input.in gives: - menu choices 0 and x; - buckets: a cancel, 11, x and 123; then 0, where nobody saw the tips, and 1.
Built on the verified corpus cases rate-numerator-outgrows-denominator (sign-ups and visitors from different populations, mismatched, clamped and matched) and usage-reported-as-effect (a feature's usage count against its true lift as exposure widens).
Recorded sessions
What the screen shows while someone uses the program. Each typed line appears after its prompt, the way a terminal shows it.
bad-input
interpreter: byte-equal== Signup funnel ==
Sign-up rates by segment, and the sign-ups credited to the onboarding tips.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 0
Pick a number from 1 to 5.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> x
Pick a number from 1 to 5.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)>
Cancelled.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 11
Type a number of buckets from 0 to 10.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> x
Type a number of buckets from 0 to 10.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 123
Type a number of buckets from 0 to 10.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 0
Shown to 0 of 10 buckets, 0 visitors: 157 sign-ups in all.
Nobody saw the tips: nothing to credit, and nothing they caused.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 1
Shown to 1 of 10 buckets, 100 visitors: 158 sign-ups in all.
by users who saw the tips: 41, 25.9% of the sign-ups
caused by the tips: 1, 0.6% of the sign-ups
bucket signed up after seeing the tips of them only because of the tips
1 41 1
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 5
Bye.
What was typed (15 lines)
0
x
3
3
11
3
x
3
123
3
0
3
1
5
basic
interpreter: byte-equal== Signup funnel ==
Sign-up rates by segment, and the sign-ups credited to the onboarding tips.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 1
segment human visitors human sign-ups bot and test visitors bot and test sign-ups
search 120 18 30 2
ads 40 4 200 50
social 80 8 60 20
email 50 15 5 5
The sign-ups come off the event table, which has them all; the visitors off a view of humans only.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 2
segment events / humans clamped humans / humans
search 16.7% 16.7% 15.0%
ads 135.0% 100.0% 10.0%
social 35.0% 35.0% 10.0%
email 40.0% 40.0% 30.0%
Events over humans: 4 of 4 segments wrong, 1 of them over 100% - the one anyone notices.
Clamped at 100%: 4 of 4 still wrong, 0 over 100%: nothing is left to notice.
Humans over humans is the rate: both numbers from the same population.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 5
Shown to 5 of 10 buckets, 500 visitors: 166 sign-ups in all.
by users who saw the tips: 139, 83.7% of the sign-ups
caused by the tips: 9, 5.4% of the sign-ups
bucket signed up after seeing the tips of them only because of the tips
1 41 1
2 33 1
3 27 2
4 21 2
5 17 3
The count is largest where the tips change least: the most engaged sign up anyway.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 3
buckets shown the tips (0 to 10)> 10
Shown to 10 of 10 buckets, 1000 visitors: 184 sign-ups in all.
by users who saw the tips: 184, 100.0% of the sign-ups
caused by the tips: 27, 14.7% of the sign-ups
bucket signed up after seeing the tips of them only because of the tips
1 41 1
2 33 1
3 27 2
4 21 2
5 17 3
6 13 3
7 10 3
8 9 4
9 7 4
10 6 4
The count is largest where the tips change least: the most engaged sign up anyway.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 4
shown sign-ups by users who saw the tips caused by the tips
0 157 0 0.0% 0 0.0%
1 158 41 25.9% 1 0.6%
2 159 74 46.5% 2 1.3%
3 161 101 62.7% 4 2.5%
4 163 122 74.8% 6 3.7%
5 166 139 83.7% 9 5.4%
6 169 152 89.9% 12 7.1%
7 172 162 94.2% 15 8.7%
8 176 171 97.2% 19 10.8%
9 180 178 98.9% 23 12.8%
10 184 184 100.0% 27 14.7%
Shown to more buckets, the count credited to the tips goes from 41 to 184, 100.0% of all sign-ups; the tips caused 27 of them.
The credited count measures who passed through the tips, not what they changed.
1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit
choice> 5
Bye.
What was typed (8 lines)
1
2
3
5
3
10
4
5
Modules
The program as written, entry module first. Each module transpiles to its own Python file, which is what eml project run executes.
main.eml(entry)
eml# P149 signup funnel: the segments, their sign-up rates three ways, the
# onboarding tips at one rollout, and at every rollout.
import funnel
def plain(s):
0 => i
len(s) => j
while i < j and s[i] == " ":
i + 1 => i
while j > i and s[j - 1] == " ":
j - 1 => j
return s[i:j]
def pad(s, n):
s => out
while len(out) < n:
out + " " => out
return out
def lpad(s, n):
s => out
while len(out) < n:
" " + out => out
return out
def pct(t):
# tenths of a percent
t % 10 => d
return str(int((t - d) / 10)) + "." + str(d) + "%"
def show_segments():
(" " + pad("segment", 9) + "human visitors human sign-ups bot and test visitors bot and test sign-ups") ^0
for s in funnel.segments():
(" " + pad(s[0], 9) + lpad(str(s[1]), 14) + lpad(str(s[2]), 16) + lpad(str(s[3]), 23) + lpad(str(s[4]), 23)) ^0
"The sign-ups come off the event table, which has them all; the visitors off a view of humans only." ^0
def rates():
(" " + pad("segment", 9) + "events / humans clamped humans / humans") ^0
0 => wrong1
0 => over1
0 => wrong2
0 => over2
for s in funnel.segments():
funnel.mismatched(s) => a
funnel.clamped(s) => b
funnel.matched(s) => c
(" " + pad(s[0], 9) + lpad(pct(a), 15) + lpad(pct(b), 10) + lpad(pct(c), 18)) ^0
if a != c:
wrong1 + 1 => wrong1
if a > 1000:
over1 + 1 => over1
if b != c:
wrong2 + 1 => wrong2
if b > 1000:
over2 + 1 => over2
len(funnel.segments()) => n
("Events over humans: " + str(wrong1) + " of " + str(n) + " segments wrong, " + str(over1) + " of them over 100% - the one anyone notices.") ^0
("Clamped at 100%: " + str(wrong2) + " of " + str(n) + " still wrong, " + str(over2) + " over 100%: nothing is left to notice.") ^0
"Humans over humans is the rate: both numbers from the same population." ^0
def read_rollout():
plain(input("buckets shown the tips (0 to 10)> ")) => t
if t == "":
return 0 - 1
if len(t) > 2:
return 0 - 2
for c in t:
if c < "0" or c > "9":
return 0 - 2
if int(t) > 10:
return 0 - 2
return int(t)
def one_rollout():
read_rollout() => r
if r == 0 - 1:
"Cancelled." ^0
return
if r < 0:
"Type a number of buckets from 0 to 10." ^0
return
funnel.rollout(r) => x
("Shown to " + str(r) + " of 10 buckets, " + str(r * 100) + " visitors: " + str(x[0]) + " sign-ups in all.") ^0
if r == 0:
" Nobody saw the tips: nothing to credit, and nothing they caused." ^0
return
(" by users who saw the tips: " + str(x[1]) + ", " + pct(funnel.tenths(x[1], x[0])) + " of the sign-ups") ^0
(" caused by the tips: " + str(x[2]) + ", " + pct(funnel.tenths(x[2], x[0])) + " of the sign-ups") ^0
(" " + pad("bucket", 8) + "signed up after seeing the tips of them only because of the tips") ^0
0 => b
for y in funnel.buckets():
if b < r:
(" " + pad(str(b + 1), 8) + lpad(str(y[0] + y[1]), 30) + lpad(str(y[1]), 35)) ^0
b + 1 => b
if r >= 2:
"The count is largest where the tips change least: the most engaged sign up anyway." ^0
def every_rollout():
(" " + pad("shown", 7) + "sign-ups by users who saw the tips caused by the tips") ^0
0 => r
while r <= 10:
funnel.rollout(r) => x
(" " + pad(str(r), 7) + lpad(str(x[0]), 8) + lpad(str(x[1]) + " " + lpad(pct(funnel.tenths(x[1], x[0])), 6), 30) + lpad(str(x[2]) + " " + lpad(pct(funnel.tenths(x[2], x[0])), 6), 21)) ^0
r + 1 => r
funnel.rollout(1) => a
funnel.rollout(10) => z
("Shown to more buckets, the count credited to the tips goes from " + str(a[1]) + " to " + str(z[1]) + ", " + pct(funnel.tenths(z[1], z[0])) + " of all sign-ups; the tips caused " + str(z[2]) + " of them.") ^0
"The credited count measures who passed through the tips, not what they changed." ^0
"== Signup funnel ==" ^0
"Sign-up rates by segment, and the sign-ups credited to the onboarding tips." ^0
True => running
while running:
"" ^0
"1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit" ^0
plain(input("choice> ")) => choice
if choice == "1":
show_segments()
elif choice == "2":
rates()
elif choice == "3":
one_rollout()
elif choice == "4":
every_rollout()
elif choice == "5":
False => running
else:
"Pick a number from 1 to 5." ^0
"Bye." ^0
Python projection (main.py)
import funnel
def plain(s):
i = 0
j = len(s)
while i < j and s[i] == " ":
i = i + 1
while j > i and s[j - 1] == " ":
j = j - 1
return s[i:j]
def pad(s, n):
out = s
while len(out) < n:
out = out + " "
return out
def lpad(s, n):
out = s
while len(out) < n:
out = " " + out
return out
def pct(t):
d = t % 10
return str(int((t - d) / 10)) + "." + str(d) + "%"
def show_segments():
print(" " + pad("segment", 9) + "human visitors human sign-ups bot and test visitors bot and test sign-ups")
for s in funnel.segments():
print(" " + pad(s[0], 9) + lpad(str(s[1]), 14) + lpad(str(s[2]), 16) + lpad(str(s[3]), 23) + lpad(str(s[4]), 23))
print("The sign-ups come off the event table, which has them all; the visitors off a view of humans only.")
def rates():
print(" " + pad("segment", 9) + "events / humans clamped humans / humans")
wrong1 = 0
over1 = 0
wrong2 = 0
over2 = 0
for s in funnel.segments():
a = funnel.mismatched(s)
b = funnel.clamped(s)
c = funnel.matched(s)
print(" " + pad(s[0], 9) + lpad(pct(a), 15) + lpad(pct(b), 10) + lpad(pct(c), 18))
if a != c:
wrong1 = wrong1 + 1
if a > 1000:
over1 = over1 + 1
if b != c:
wrong2 = wrong2 + 1
if b > 1000:
over2 = over2 + 1
n = len(funnel.segments())
print("Events over humans: " + str(wrong1) + " of " + str(n) + " segments wrong, " + str(over1) + " of them over 100% - the one anyone notices.")
print("Clamped at 100%: " + str(wrong2) + " of " + str(n) + " still wrong, " + str(over2) + " over 100%: nothing is left to notice.")
print("Humans over humans is the rate: both numbers from the same population.")
def read_rollout():
t = plain(input("buckets shown the tips (0 to 10)> "))
if t == "":
return 0 - 1
if len(t) > 2:
return 0 - 2
for c in t:
if c < "0" or c > "9":
return 0 - 2
if int(t) > 10:
return 0 - 2
return int(t)
def one_rollout():
r = read_rollout()
if r == 0 - 1:
print("Cancelled.")
return
if r < 0:
print("Type a number of buckets from 0 to 10.")
return
x = funnel.rollout(r)
print("Shown to " + str(r) + " of 10 buckets, " + str(r * 100) + " visitors: " + str(x[0]) + " sign-ups in all.")
if r == 0:
print(" Nobody saw the tips: nothing to credit, and nothing they caused.")
return
print(" by users who saw the tips: " + str(x[1]) + ", " + pct(funnel.tenths(x[1], x[0])) + " of the sign-ups")
print(" caused by the tips: " + str(x[2]) + ", " + pct(funnel.tenths(x[2], x[0])) + " of the sign-ups")
print(" " + pad("bucket", 8) + "signed up after seeing the tips of them only because of the tips")
b = 0
for y in funnel.buckets():
if b < r:
print(" " + pad(str(b + 1), 8) + lpad(str(y[0] + y[1]), 30) + lpad(str(y[1]), 35))
b = b + 1
if r >= 2:
print("The count is largest where the tips change least: the most engaged sign up anyway.")
def every_rollout():
print(" " + pad("shown", 7) + "sign-ups by users who saw the tips caused by the tips")
r = 0
while r <= 10:
x = funnel.rollout(r)
print(" " + pad(str(r), 7) + lpad(str(x[0]), 8) + lpad(str(x[1]) + " " + lpad(pct(funnel.tenths(x[1], x[0])), 6), 30) + lpad(str(x[2]) + " " + lpad(pct(funnel.tenths(x[2], x[0])), 6), 21))
r = r + 1
a = funnel.rollout(1)
z = funnel.rollout(10)
print("Shown to more buckets, the count credited to the tips goes from " + str(a[1]) + " to " + str(z[1]) + ", " + pct(funnel.tenths(z[1], z[0])) + " of all sign-ups; the tips caused " + str(z[2]) + " of them.")
print("The credited count measures who passed through the tips, not what they changed.")
print("== Signup funnel ==")
print("Sign-up rates by segment, and the sign-ups credited to the onboarding tips.")
running = True
while running:
print("")
print("1) segments 2) rates 3) the tips at one rollout 4) every rollout 5) quit")
choice = plain(input("choice> "))
if choice == "1":
show_segments()
elif choice == "2":
rates()
elif choice == "3":
one_rollout()
elif choice == "4":
every_rollout()
elif choice == "5":
running = False
else:
print("Pick a number from 1 to 5.")
print("Bye.")
funnel.eml
eml# P149 signup funnel - sign-up rates by traffic segment, and a feature
# credited with sign-ups. As in the corpus case
# rate-numerator-outgrows-denominator, the sign-ups come off the event table,
# which has every sign-up, and the visitors off a curated view of humans: the
# ratio of two right numbers is not a rate, and clamping the one that goes
# over 100% turns a loud impossible number into a quiet wrong one. As in the
# corpus case usage-reported-as-effect, "sign-ups by users who saw the tips"
# counts what passed through the feature, not what it changed: it grows as
# the feature is shown to more people, and is largest where the feature
# changes least.
def segments():
# [name, human visitors, human sign-ups, bot and test visitors, bot and
# test sign-ups]
return [["search", 120, 18, 30, 2], ["ads", 40, 4, 200, 50], ["social", 80, 8, 60, 20], ["email", 50, 15, 5, 5]]
def tenths(n, d):
# n / d as a percentage in tenths, rounded half up
return int((2000 * n + d) / (2 * d))
def mismatched(s):
# every sign-up over the human visitors
return tenths(s[2] + s[4], s[1])
def clamped(s):
if mismatched(s) > 1000:
return 1000
return mismatched(s)
def matched(s):
# both from the same population: humans over humans
return tenths(s[2], s[1])
def buckets():
# ten buckets of 100 visitors, the most engaged first, the feature shown
# to the first ones: [would sign up anyway, sign up only with the tips]
return [[40, 1], [32, 1], [25, 2], [19, 2], [14, 3], [10, 3], [7, 3], [5, 4], [3, 4], [2, 4]]
def rollout(r):
# the tips shown to the first r buckets: [sign-ups in all, sign-ups by
# users who saw the tips, sign-ups the tips caused]
0 => total
0 => saw
0 => caused
0 => b
for x in buckets():
total + x[0] => total
if b < r:
total + x[1] => total
saw + x[0] + x[1] => saw
caused + x[1] => caused
b + 1 => b
return [total, saw, caused]
Python projection (funnel.py)
def segments():
return [["search", 120, 18, 30, 2], ["ads", 40, 4, 200, 50], ["social", 80, 8, 60, 20], ["email", 50, 15, 5, 5]]
def tenths(n, d):
return int((2000 * n + d) / (2 * d))
def mismatched(s):
return tenths(s[2] + s[4], s[1])
def clamped(s):
if mismatched(s) > 1000:
return 1000
return mismatched(s)
def matched(s):
return tenths(s[2], s[1])
def buckets():
return [[40, 1], [32, 1], [25, 2], [19, 2], [14, 3], [10, 3], [7, 3], [5, 4], [3, 4], [2, 4]]
def rollout(r):
total = 0
saw = 0
caused = 0
b = 0
for x in buckets():
total = total + x[0]
if b < r:
total = total + x[1]
saw = saw + x[0] + x[1]
caused = caused + x[1]
b = b + 1
return [total, saw, caused]