Affine map
A transformation that applies a linear map and then adds a fixed offset; it preserves …
Attention mask
A model or library rule that marks usable positions or specifies which sequence positions …
Base-rate bias
A low absolute probe-token rank can reflect its prompt-conditioned baseline or a …
Best-rank
The minimum vocabulary rank a pre-specified probe token hits across layers, positions, and …
BF16, FP16, and FP32
Floating-point formats for model arithmetic: 32-bit floating point (FP32), standard 16-bit …
Bootstrap
A resampling method that repeatedly draws observed units with replacement to study how a …
Brier loss
The average squared gap between a stated probability and the zero-or-one outcome that …
Cosine similarity
A comparison of two nonzero vectors based on the angle between them rather than their …
Data leakage
An accidental flow of information across an evaluation boundary that makes held-out …
Dtype
The data type (dtype) of a tensor or array: the storage format shared by every element, …
Embedding
A vector of numbers used to represent an item so a model or retrieval system can compare …
Feed-forward network (FFN)
A computation with no feedback cycle; in a Transformer, usually the shared position-wise …
Fisher's exact test
An exact conditional test for association in a count table, commonly used for a 2 by 2 …
Gini impurity
A measure of how mixed the class labels are inside a group of rows, used by decision trees …
Greedy continuation
The model's generated text when each next token is chosen as the single …
Hidden bridge
An intermediate entity that logically links a two-hop question to its answer while …
i.i.d.
Independent and identically distributed: each draw is independent of the others and from …
Imputation
A documented rule for filling missing values so an analysis can use a complete table …
Jacobian lens
A corpus-averaged linear map that turns a mid-layer residual-stream state into …
L2 normalization
Rescaling a nonzero vector to Euclidean length one while preserving its direction.
Log loss
The average negative logarithm of the probability a model gave to what actually happened, …
Logit lens
The free mid-layer readout that skips any fitted transport: unembed the residual as if it …
Matmul
Matrix multiplication: the core linear-algebra operation behind attention, multilayer …
Mean pooling
A rule that averages selected vectors to produce one summary vector.
Motor layer / motor-late
A late, near-output regime; in one scoped 397B analysis, digit-direction vocabulary-score …
Multilayer perceptron (MLP)
The position-wise nonlinear feed-forward sublayer that transforms each token state and …
p-value
A tail probability under a stated null model: how often that null produces a result at …
Pearson correlation
A number from negative one to positive one that summarizes the direction and strength of a …
Principal component analysis (PCA)
A linear dimensionality-reduction method that replaces numerical feature columns with new …
Prompt echo
When a lens lights up words that were already in the prompt; looks like a finding, often …
Random-J
A family of randomized Jacobian-lens controls; each construction states which scale or …
Rank
Position of a target token ID in a scored vocabulary list (1 = highest-scoring). Lower is …
Relevance judgments
Reference labels that say which documents are considered relevant to each evaluation …
Residual stream
The model's running hidden-state channel: each layer reads it, adds an update, and passes …
Retrieval ranking metrics
Measures that compare a ranked retrieval list with relevance judgments, including …
Root mean square (RMS)
The square root of the mean squared coordinates of a vector, used here as a measure of …
Sampling
Choosing the next token from a distribution over the vocabulary, rather than always taking …
Standard deviation
A measure of how far values in one numerical feature typically spread around their mean.
Standard scaling
A feature-wise transformation that subtracts each column's mean and divides by that …
Survival-identity vocabulary
The frozen probe list for the primary self-vs-other-model battery; its exact normalized …
Temperature
A scalar that sharpens or flattens the next-token distribution before sampling.
Thinking on / off
The template-controlled reasoning protocols used in these notes: reasoning enabled, or …
Tokenization
The model-specific process that divides an input into tokens and maps those tokens to …
Top-p
Nucleus sampling: keep only the smallest set of tokens whose cumulative probability …
Transformer
A family of sequence-processing neural networks that combine attention, position-wise …
Unembedding
The final linear map from residual-stream coordinates to a score for every vocabulary …
Wilcoxon signed-rank test
A paired test that uses each gap's direction and its absolute-size rank, not just win …
Workspace
A workspace-like mid-network regime studied through selected residual directions and …