lstm

std.nn.lstm · Level L3

Run an LSTM over a sequence of inputs; returns every hidden state and every cell state. A Scan of lstm_cell with two carries, h and c (Round 14.10), differentiable end to end.

(hₜ, cₜ) = lstm_cell(hₜ₋₁, cₜ₋₁, xₜ)

Signature

lstm(h0: f64[k], c0: f64[k], X: f64[n, m], W: f64[4, k, k], U: f64[4, k, m], b: f64[4, k]) → f64[n, k], f64[n, k]

Structure

The function as NOVA stores it: one box per input, operation and output, and arrows that carry values. A double border marks another library function this one runs — called once, or by Scan once per element; select it to open that function.

input
operation
constant
call
output

Verification

Signature proven by NOVA’s shape solver, for every size.
Agrees with the reference h, c = h0, c0; for each x in X: h, c = lstm_cell(h, c, x) to 80 digits (100-digit arithmetic), on all 40 test cases.
All 1,052 float64 results inside the running error bound; the closest uses 14% of it.
Interpreter and NumPy backend return bit-identical results.
At float32 too: the 40 test cases rounded to float32, and 40 of them inside the running error bound at u = 2⁻²⁴; the closest uses 43% of it.

Accuracy in detail

correctly rounded (the float64 nearest the exact value): 54%
bit-equal to the NumPy formula in float64: 100%
largest error, in units in the last place: 212

Large ulp counts appear where a result is tiny next to the numbers it is computed from (after cancellation, for example), so one unit in the last place is tiny too; the absolute error is still inside the bound. Results within their own error of zero are not counted.

Identity

Calls

lstm_cell

Called by

—

sha256:7593a29978b6d1fcdc546f0bb0e54db55ace65e66f3fb3e8799084cc6014f98b

The semantic hash of the graph. It changes when the program changes, and never when only its documentation does.

Control handle

Symbol: Ω:std.nn.lstm · Ω:lstm
Pins: sha256:2a02ebb147a10e83a3d6145c6689c590897b0c976cbeca177b00a11b66a9c8dbthis graph and the 1 it reaches through calls
Evidence: sha256:e9032b055a13b916a2bffe8d91d612153cec3f6bc2e2f4ba48109fb1d1f88c58the hash of its verification record
Needs: no capability: a pure function

Through NOVA’s control layer, the symbol launches this function only while the program still matches what it pins: a change to this graph, or to any graph it reaches, needs a migration first.