adam_step

std.optim.adam_step · Level L2

One step of Adam (Kingma and Ba) on the state [params; m; v] at step t (1, 2, …): m ← β₁m + (1 − β₁)g, v ← β₂v + (1 − β₂)g², then params ← params − lr·m̂/(√v̂ + ε) with the bias corrections m̂ = m/(1 − β₁ᵗ), v̂ = v/(1 − β₂ᵗ).

m ← β₁m + (1−β₁)g, v ← β₂v + (1−β₂)g², θ ← θ − η·(m/(1−β₁ᵗ)) / (√(v/(1−β₂ᵗ)) + ε)

Signature

adam_step(state: f64[3, n], grad: f64[n], t: f64[], lr: f64[], beta1: f64[], beta2: f64[], eps: f64[]) → f64[3, n]

Structure

The function as NOVA stores it: one box per input, operation and output, and arrows that carry values. A double border marks another library function this one runs — called once, or by Scan once per element; select it to open that function.

input
operation
constant
call
output

Verification

Signature proven by NOVA’s shape solver, for every size.
Agrees with the reference Adam's update written out element by element to 80 digits (100-digit arithmetic), on all 40 test cases.
All 513 float64 results inside the running error bound; the closest uses 86% of it.
Interpreter and NumPy backend return bit-identical results.
At float32 too: the 40 test cases rounded to float32, and 40 of them inside the running error bound at u = 2⁻²⁴; the closest uses 93% of it.

Accuracy in detail

correctly rounded (the float64 nearest the exact value): 69%
bit-equal to the NumPy formula in float64: 89%
largest error, in units in the last place: 377

Large ulp counts appear where a result is tiny next to the numbers it is computed from (after cancellation, for example), so one unit in the last place is tiny too; the absolute error is still inside the bound. Results within their own error of zero are not counted.

Identity

Calls

—

Called by

adam_descent_step logistic_adam_step

sha256:431082dfc67e29472c4dbac508698ff2758564edc7ec4aba775c329cba7e39ec

The semantic hash of the graph. It changes when the program changes, and never when only its documentation does.

Control handle

Symbol: Ω:std.optim.adam_step · Ω:adam_step
Pins: sha256:7e6924652bd08f16a88d2e2763dfc3ba155c430961c561da553c88996ec6e618this graph alone
Evidence: sha256:95444dc8a04a01a5b488b44d9f80f0f3083d11f42d3501051c2e1165b8e2a719the hash of its verification record
Needs: no capability: a pure function

Through NOVA’s control layer, the symbol launches this function only while the program still matches what it pins: a change to this graph, or to any graph it reaches, needs a migration first.