EML-NOVA · Primitives
What the graphs are made of.
Every library function is composed from these primitives. The last column counts the functions that use each one directly.
| Primitive | What it does | Inputs | Differentiable | Used by |
|---|---|---|---|---|
| Constant | A literal value inside the graph. | 0 | yes | 311 |
| Add | Elementwise sum, NumPy-style broadcasting. | 2 | yes | 183 |
| Subtract | Elementwise difference, broadcasting. | 2 | yes | 243 |
| Multiply | Elementwise product, broadcasting. | 2 | yes | 315 |
| Divide | Elementwise quotient, broadcasting. | 2 | yes | 81 |
| Negate | Elementwise negation. | 1 | yes | 42 |
| MatMul | Matrix product over the last two axes; leading axes broadcast. | 2 | yes | 58 |
| Transpose | Permute the axes. | 1 | yes | 49 |
| Reshape | Same elements, new shape; one −1 is inferred. | 1 | yes | 136 |
| ReduceSum | Sum along one axis, or over everything. | 1 | yes | 137 |
| Mean | Mean along one axis, or over everything. | 1 | yes | 22 |
| Relu | max(x, 0), elementwise. | 1 | yes | 6 |
| Sigmoid | 1 / (1 + e⁻ˣ), elementwise. | 1 | yes | 4 |
| Tanh | Hyperbolic tangent, elementwise. | 1 | yes | 4 |
| Softmax | Normalised exponentials along one axis. | 1 | yes | 1 |
| Identity | Pass a value through unchanged. | 1 | yes | 32 |
| StopGradient | Identity going forward, zero gradient going back. | 1 | yes | — |
| If | Choose between two values by a scalar condition. | 3 | not yet | — |
| BoundedLoop | A loop with a fixed bound; for now its body is one elementwise operation. | 1 | not yet | — |
| Call | Call another graph of the same module. This is how the library composes; since Round 13.1, differentiation follows the call into the callee. | any | yes | 171 |
| Sqrt | Square root, elementwise; correctly rounded (IEEE 754). | 1 | yes | 28 |
| Exp | Exponential, elementwise. | 1 | yes | 20 |
| Log | Natural logarithm, elementwise. | 1 | yes | 21 |
| Maximum | Exact elementwise maximum, broadcasting; ties send the gradient to the first operand. | 2 | yes | 62 |
| Minimum | Exact elementwise minimum, broadcasting; ties send the gradient to the first operand. | 2 | yes | 28 |
| Less | Elementwise a < b as a 0/1 float mask; zero gradient. | 2 | yes | 53 |
| Greater | Elementwise a > b as a 0/1 float mask; zero gradient. | 2 | yes | 91 |
| Equal | Elementwise a = b as a 0/1 float mask; zero gradient. | 2 | yes | 154 |
| Where | Select a where the condition is non-zero, else b; all three broadcast. | 3 | yes | 53 |
| ReduceMax | Largest value along one axis, or overall; ties share the gradient equally. | 1 | yes | 76 |
| ReduceMin | Smallest value along one axis, or overall; ties share the gradient equally. | 1 | yes | 56 |
| Size | The element count of a tensor, or the length of one axis, as a value; zero gradient. | 1 | yes | 74 |
| Slice | A contiguous window along one axis; negative bounds count from the end. A window outside the axis is refused, not clamped. | 1 | yes | 94 |
| Concat | Join tensors along one axis. | any | yes | 57 |
| Iota | The positions 0, 1, …, n−1 along one axis, as values shaped to broadcast; zero gradient. | 1 | yes | 99 |
| Sort | Stable sort along one axis, ascending or descending; the gradient follows the permutation back. | 1 | yes | 4 |
| ArgMax | Position of the first largest value, as a value; zero gradient. | 1 | yes | 19 |
| ArgMin | Position of the first smallest value, as a value; zero gradient. | 1 | yes | 3 |
| CumSum | Running sum along one axis, from the start or from the end. | 1 | yes | 3 |
| CumMax | Running maximum along one axis; ties keep the earlier position. | 1 | yes | 1 |
| CumMin | Running minimum along one axis; ties keep the earlier position. | 1 | yes | 1 |
| Scan | A loop whose body is another graph: carry a value through a sequence, one step per element, and keep every step. Differentiable: the gradient runs the loop backwards. | any | yes | 54 |
| Cast | Convert between float64 and int64, exactly: a value that is not a whole number, or lies beyond ±2⁵³, is refused, never rounded; zero gradient. | 1 | yes | 81 |
| Gather | Pick slices along an axis by integer indices (np.take); an index outside the axis is refused. The gradient is scattered back. | 2 | yes | 71 |
| GatherAlong | Pick one element per position along an axis (np.take_along_axis): per-row labels, orders from another array. | 2 | yes | 4 |
| ScatterAdd | Add slices into a base at integer indices along an axis; repeated indices accumulate (np.add.at). | 3 | yes | 3 |
| ArgSort | The stable sorting permutation, as positions (floats, like ArgMax); Cast them to index with them. Zero gradient. | 1 | yes | 3 |
| While | A loop that runs a body graph while a condition graph says go on, up to a stated maximum number of steps; exceeding it is an error, not a truncated answer. Differentiable: the gradient replays the steps taken. | any | yes | 26 |
| Solve | Solve A·x = b by Gaussian elimination with partial pivoting; a singular matrix is refused. | 2 | yes | 5 |
| Cholesky | The lower-triangular L with L·Lᵀ = A for a symmetric positive-definite A (reads A's lower triangle). | 1 | yes | 5 |
| TriangularSolve | Forward or back substitution with a triangular matrix. | 2 | yes | 4 |
| EighValues | The eigenvalues of a symmetric matrix, ascending. | 1 | yes | 7 |
| EighVectors | The unit eigenvectors of a symmetric matrix, each signed so its largest component is positive. | 1 | yes | 5 |
| QROrthogonal | The orthonormal factor Q of A = Q·R (Householder QR) for a matrix with at least as many rows as columns. | 1 | yes | 3 |
| QRTriangular | The upper-triangular factor R of A = Q·R, with a positive diagonal; a matrix without full column rank is refused. | 1 | yes | 2 |
| SVDValues | The singular values of a matrix, largest first. | 1 | yes | 7 |
| SVDLeftVectors | The left singular vectors, as columns, each pair signed by its right vector. | 1 | yes | 3 |
| SVDRightVectors | The right singular vectors, as columns, each signed so its largest component is positive. | 1 | yes | 4 |
| FloorDiv | ⌊a/b⌋, rounded toward −∞: exact for integers, the floor of the exact quotient for floats; a zero divisor is refused. | 2 | yes | 28 |
| Mod | a − b·⌊a/b⌋, the remainder with the divisor's sign. | 2 | yes | 29 |
| BitAnd | Bitwise and of two int64 tensors, elementwise (Python's &). Exact; a float operand is refused. | 2 | not yet | — |
| BitOr | Bitwise or of two int64 tensors, elementwise (Python's |). Exact; a float operand is refused. | 2 | not yet | — |
| BitXor | Bitwise exclusive or of two int64 tensors, elementwise (Python's ^): what Philox4x32-10 mixes its words with. Exact; a float operand is refused. | 2 | not yet | 1 |
| Sin | Sine, elementwise, in radians. The argument is reduced exactly: measured within 1.2 units of roundoff of float64 for arguments up to 10³⁰⁰. | 1 | yes | 7 |
| Cos | Cosine, elementwise, in radians. | 1 | yes | 6 |
| Tan | Tangent, elementwise, in radians; poles where cos x = 0, on which no float lies. | 1 | yes | — |
| Asin | Arcsine, elementwise: the angle in [−π/2, π/2] whose sine is x, for x in [−1, 1]. | 1 | yes | — |
| Acos | Arccosine, elementwise: the angle in [0, π] whose cosine is x, for x in [−1, 1]. | 1 | yes | 3 |
| Atan | Arctangent, elementwise: the angle in (−π/2, π/2) whose tangent is x. | 1 | yes | — |
| Atan2 | The angle of the point (x, y), elementwise: atan2(y, x) in [−π, π], stable in every quadrant; on the negative x-axis the sign of a zero y decides ±π (IEEE 754). | 2 | yes | 4 |
| FFT | The discrete Fourier transform of a complex vector stored as [2, n] (real parts, imaginary parts), unnormalized. | 1 | yes | 4 |
| IFFT | The inverse transform, with its 1/n, so IFFT(FFT(z)) = z. | 1 | yes | 3 |
| Erf | The error function, erf x = (2/√π)·∫₀ˣ e^(−t²) dt. | 1 | yes | 1 |
| Erfc | The complementary error function 1 − erf x, computed directly so the tail keeps its relative accuracy. | 1 | yes | 3 |
| LogGamma | log|Γ(x)|; the poles at 0, −1, −2, … are refused. Its gradient is the digamma function. | 1 | yes | 7 |
| Expm1 | eˣ − 1, accurate near 0 where exp(x) − 1 loses every digit. | 1 | yes | 3 |
| Log1p | log(1 + x), accurate near 0 where log(1 + x) would round 1 + x first. | 1 | yes | 9 |
| Ndtri | The standard normal quantile Φ⁻¹(p), for 0 < p < 1; computed by NOVA (Acklam, refined by Halley steps). | 1 | yes | 4 |
| GammaInc | The regularized lower incomplete gamma function P(a, x); differentiable in x and in a (∂P/∂a is computed by NOVA, through the same series and continued fraction). | 2 | yes | 3 |
| GammaIncC | The regularized upper incomplete gamma function Q(a, x) = 1 − P(a, x), computed directly so the upper tail keeps its relative accuracy: survival functions and p-values far below 10⁻¹⁶, which 1 − P would round to 0. Differentiable in x and in a. | 2 | yes | 5 |
| BetaInc | The regularized incomplete beta function I_x(a, b); differentiable in x, a and b (∂I/∂a and ∂I/∂b are computed by NOVA, through the same continued fraction). | 3 | yes | 10 |
| GammaIncInv | The inverse of P(a, x) in x: the x with P(a, x) = p, for 0 ≤ p < 1; computed by NOVA (Newton on the log tail); differentiable in p and in a (by implicit differentiation). | 2 | yes | 2 |
| BetaIncInv | The inverse of I_x(a, b) in x: the x with I_x(a, b) = p; computed by NOVA (Newton on the log tail, in logit x); differentiable in p, a and b (by implicit differentiation). | 3 | yes | 2 |