A Linear Nudge, a Nonlinear Wake

Table: the frozen-panel linearity census, as ranges across the 24 prompt × direction cells. How each statistic is built and why these thresholds were chosen is in How we scored pass versus fail; Figure 1 plots every cell. Provenance: fig-1-linearity-census.receipt.json.

Study status: complete. Audited result release fde24e9; experiment package experiments/consciousness_sae_signed_dose_scan; shipping table, sample records, and hashes in the release inventory. Corrected 4 September 2026 and reorganized 2 October 2026 with no change to any result; both entries are in Appendix D.

The Question

Activation steering changes a model from the inside. In the additive form studied here, the researcher adds a controlled vector to the hidden state carried between Transformer blocks instead of changing the prompt; contrastive activation addition is the canonical modern example (Rimsky et al., 2024). The field also includes adaptive, feature-targeted, and geometry-preserving variants, which this note does not test. The added vector can encode a proposed semantic direction, such as a sparse-autoencoder feature, or it can be deliberately generic, as in the experiment reported here.

A Jacobian is a local derivative: the high-dimensional analogue of a tangent line. A Jacobian lens (Gurnee et al., 2026) turns that idea into a reusable reference by averaging input-output Jacobians over many prompts and token positions. The result is one fixed linear map for each source layer: a corpus-average first-order approximation, not the exact Jacobian of the prompt under study. Gurnee et al. built the lens to surface verbalizable, vocabulary-oriented content; this note repurposes it as a transport benchmark for arbitrary residual perturbations, a different job from the one it was fit for. Whether averaged first-order maps carry useful structure is itself setting-dependent: relation-decoding work finds that mean Jacobian-based affine maps decode some relations well and others poorly (Hernandez et al., 2024).

A downstream change therefore has at least three candidate causes: an edit that did not land as requested in low-precision arithmetic, propagation as the linear map predicts, or reshaping by the remaining nonlinear blocks. The design below records each separately, so that none is attributed to the meaning of a steering direction by default.

Prior work, contribution, and non-claims

Prior work. Gurnee et al. introduce the Jacobian lens and connect mid-layer directions to later residual and vocabulary geometry (2026). Public fitting tooling and Llama maps come from Anthropic’s reference implementation (2026) and Neuronpedia’s Llama release (2026). Recent dose and steering work reports that intervention strength need not grow effects monotonically (Taimeskhanov, Vaiter, and Garreau, 2026) and that stronger SAE refusal steering can coincide with collateral over-refusal (O’Brien et al., 2025). An earlier Praxagent note asks whether SAE steering leaves a detectable J-space fingerprint under matched access models (Jones, 2026). This note does not invent the lens or the general fact that neural networks are nonlinear; its contribution is the point-by-point separation below. The dose and collateral-effect papers cited here motivate a dose scan; they are study-specific reports, not evidence for the result in this note.

This note’s contribution.

• a prospectively frozen, target-masked signed dose scan on Llama 3.3 70B that retains requested edit, BF16-realized edit, fixed-J reference, and actual state separately at every dose and depth;
• a complete 24-cell census on the frozen 2%/3%/4% linearity panel, with identity and five random-J controls on the primary predictive comparison;
• a public ledger in tdj28/llm_selfref_pre with freeze and audited-release commits separated.

Not claimed.

• that SAE decoder directions, as a class, are nonlinear downstream; neither this scan nor the pilot in Appendix B measured downstream dose response for an SAE decoder direction (the vector associated with one learned sparse-autoencoder feature);
• that any specific SAE feature has its published semantic interpretation;
• that generated behavior, deception, or consciousness-related vocabulary changed;
• that the corpus-average J outperforms simply carrying the residual change forward (it did not clear that frozen margin here).

Design in Brief

We tested this in Llama 3.3 70B Instruct. Immediately after transformer block 50, we applied three stress-test directions to eight prompts, varied their magnitude from 0.5% to 30% of residual RMS, and followed the response through the remaining 29 blocks. The directions were unselected: independent Gaussian vectors normalized to unit RMS, generated from predeclared seeds and frozen before the experiment, with no step that picked them for meaning, expected effect, or resemblance to any SAE feature. They test the mechanics of perturbation transport rather than any concept.

What one measurement is

A “run” in this study is one deterministic forward pass, not a generation. The same network with the same weights reads the same fixed prompt; no tokens are sampled, so no temperature is involved anywhere. We record the hidden state the model computes at one fixed position (the continuation token, the final rendered prompt token) after each block.

For each prompt there is one clean run with no intervention. For each direction and dose there are two edited runs, identical to the clean run in every respect except one: immediately after block 50, the direction vector, scaled to the requested dose, is added to (\(+b\) run) or subtracted from (\(-b\) run) the residual state at that position before block 51 continues. The model is therefore its own control. Any difference between runs at any later block can only have come from that one injected vector, because nothing else differs: weights, prompt, position, and arithmetic path are identical. The clean run also sets the dose scale: a dose of 2% means the injected vector’s RMS is 2% of the clean run’s layer-50 residual RMS for that prompt. The frozen grid comes to 2,896 forward passes in total.

The paired edited runs act like pushing and then pulling the same knob. Any downstream change genuinely driven by the edit should flip sign between the two runs; anything that shows up either way (noise, or the model reacting to being disturbed at all) should not. Half the difference between the two runs keeps the flipping part and cancels the rest; we call that the signed response (the implementation’s name for it is the central contrast). The non-flipping part, measured as the midpoint of the two runs relative to an unedited run, is kept as a separate common-mode diagnostic. Because each \(+\)/\(-\) pair collapses into one curve, the census has 24 prompt × direction curves rather than 48 independent observations. Exact construction, provenance, dose grid, and pairing equations are in Appendix A.

Source: this research note.

Diagram: how one measurement is built from three forward passes. Hand-drawn conceptual schematic, not a receipt-backed data figure; the exact pairing equations are in Appendix A.

At every dose, the analysis retained four quantities: the edit requested in high precision; the edit realized after casting into BF16 activations (see requested-versus-realized fidelity); the downstream change from the fixed Jacobian map; and the downstream change the model actually produced. Separating requested from realized keeps low-precision rounding from being mistaken for nonlinear model dynamics. The full decomposition is drawn as a schematic in Appendix A.

Every decision rule (which doses count, which quantities are compared, and the numeric thresholds for passing) was written down and committed to git before any results were computed (a084caa). The ordering matters: thresholds chosen after seeing the data can make a “pass” or “fail” an artifact of where the goalposts were placed, whereas rules frozen first let the experiment come out either way. Concretely, the frozen rule says: once a small edit demonstrably landed intact, the model’s final response counts as linear only if scaling the dose across 2%/3%/4% scales the response by the same factor while keeping nearly the same direction, within the two numeric bounds defined below. Failing that rule means the local-linear description broke by a prespecified margin on this panel; it does not mean “networks are nonlinear” in general, and it is not an estimate of how often this happens across other models or directions.

The Linear Edit and the Nonlinear Wake

What “linear” means in this note

Imagine a dimmer switch. If the response is dose-linear, turning the dose from 2% to 4% should roughly double the size of the change while leaving its direction alone. In vector language: the hidden-state response at dose \(b\) should look like \(b\cdot v\) for one fixed direction \(v\).

Nonlinear, here, means that rule fails in a measured way: as dose changes, the response rotates (direction drifts) or its size stops tracking the dose (amplification or compression). We are not using “nonlinear” as a vibe word for “the network has nonlinear layers.” Transformers are nonlinear by construction. The question is whether this dose–response curve stayed close enough to a straight line through the origin on a small, prechosen panel.

A cell is one prompt paired with one generic direction: 8 prompts × 3 directions = 24 cells. Each cell is one signed-response curve across dose, so the census has 24 curves, not 48 independent signed runs.

We apply the same linearity test to three quantities in each cell:

1. Source: the edit that actually landed after block 50.
2. Fixed J: that same realized edit pushed through the corpus-average Jacobian map (an algebraic straight-line reference).
3. Actual final: the model’s real residual change after block 79.

How we scored pass versus fail

The frozen panel is \(B=\{0.02,0.03,0.04\}\) (2%, 3%, 4%), with 3% as the anchor. That panel is the first three-dose window where every source edit already cleared the requested-versus-realized fidelity checks; Figure 2 shows that boundary.

Every response vector \(x_b\) below is the signed response from the design section, read at a different point: right after block 50 for the source, after block 79 for the actual final, and after pushing the source vector through the released linear map for fixed J. No new runs, prompts, or sampling are involved; the three quantities are three readouts of the same 2,896 deterministic forward passes.

For a response vector \(x_b\) at dose \(b\), form the dose-normalized vector

The divisor \(s_b\) is the size of the input being scaled. For the source edit, \(s_b\) is the requested dose fraction (0.02, 0.03, 0.04). For J and the actual final state, \(s_b\) is instead the measured RMS size of the edit that actually landed at block 50, expressed as the same kind of fraction. The reason for the switch: those two quantities are downstream of the realized edit, so the fair question is “per unit of edit that landed, what came out?”, not “per unit of edit we asked for.” Dividing by \(s_b\) puts every dose on that per-unit scale. If doubling the landed edit exactly doubled the downstream response without turning it, the normalized vectors \(q_b\) at 2%, 3%, and 4% would all be the same vector; without the division, even a perfectly linear response \(x_b=b\,v\) would change length across the panel. How far the \(q_b\) spread apart is what the two statistics below measure.

Two statistics then ask whether the \(q_b\) stay near the 3% anchor \(q_{0.03}\):

• \(c_{\min}\) is directional agreement (1 means identical direction).
• \(d_{\max}\) is relative size mismatch after dose normalization (0 means identical scale and shape). The implementation calls \(d_{\max}\) “slope discrepancy.”

A cell passes for a quantity when both frozen gates hold:

Fail either gate and that quantity is scored nonlinear on this panel. Exact construction of \(x_b\) (signed branches, common-mode diagnostic) is in Appendix A.

import numpy as np

B = (0.02, 0.03, 0.04)
ANCHOR = 0.03

def rms(x: np.ndarray) -> float:
    return float(np.sqrt(np.mean(np.square(x))))

def cosine(a: np.ndarray, b: np.ndarray) -> float:
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

def dose_linearity_gates(
    x_by_dose: dict[float, np.ndarray],
    s_by_dose: dict[float, float],
    *,
    cosine_min: float = 0.95,
    slope_max: float = 0.15,
) -> tuple[float, float, str]:
    """Return (c_min, d_max, 'pass'|'fail') for one cell and one quantity."""
    q = {b: x_by_dose[b] / s_by_dose[b] for b in B}
    q_anchor = q[ANCHOR]
    c_min = min(cosine(q[b], q_anchor) for b in B)
    d_max = max(rms(q[b] - q_anchor) / rms(q_anchor) for b in B)
    status = "pass" if c_min >= cosine_min and d_max <= slope_max else "fail"
    return c_min, d_max, status

# Example call for one prompt × direction cell:
# c_min, d_max, status = dose_linearity_gates(source_x, source_s)

What the census found

The table at the head of this note is the result: 24 of 24 source edits passed both gates, 24 of 24 fixed-J references passed, and 0 of 24 actual final-state responses did. The finals did not fail by a hair on one metric. In every cell they failed both rules: the dose-normalized response rotated away from the 3% anchor (cosine well below 0.95) and its per-unit-dose size and shape drifted far past the 0.15 ceiling, while source and fixed-J curves sat near cosine 1 with only about 0.1 relative discrepancy. After 29 more blocks, turning the dose from 2% to 4% no longer gave “same arrow, twice as long”: the arrow bent, and its length stopped tracking dose.

The fixed-J pass is a consistency reference, not a prediction. A fixed linear map preserves exact proportionality (if \(x_b=bv\), then \(Jx_b=bJv\)), but for approximately proportional inputs a matrix can amplify the deviations, so passage through a finite cosine/RMS gate is empirical rather than guaranteed; here the realized edits and their J projections both passed. That locates the departure from proportionality downstream of the realized edit, in the BF16 execution of blocks 51–79 from the post-block-50 state to the post-block-79 residual, not in dose-dependent mangling of the source edit. The scan does not separate the model’s nonlinear dynamics from finite-precision effects across those blocks; a datatype (FP32/FP16) and hook-path control is the test that would (see Next experiments). Whether J predicted the model accurately is a different question, tested in Figure 4.

Figure 1 shows the complete census rather than an average. Its three horizontal categories are the realized source edit, J applied to that realized edit, and the actual final state. The left panel plots \(c_{\min}\) against the 0.95 floor; the right plots \(d_{\max}\) against the 0.15 ceiling. Each connecting line is one prespecified prompt × direction cell.

Source: this research note.

Figure 1: source linearity versus actual downstream curvature. Each line is one of 24 prompt × direction signed-response curves over the prespecified 2%/3%/4% panel; dashed lines mark the frozen thresholds. Provenance: receipt.

Requested-versus-realized edit fidelity under BF16

“Delivery” is not a standard ML term. It is this study’s short name for a simple fidelity check: does the intervention vector we requested match the vector that was realized in the residual after casting into the model’s number format?

That format here is BF16 (bfloat16): a 16-bit floating-point type with FP32-like range but much coarser precision, so many float32 values round to the same BF16 number. For bit layouts, casting demos, and accumulation swamping, see the Deep Dive BF16, FP16, and FP32: Precision, Range, Swamping, and Determinism. The protocol builds the requested edit in float32 along a unit-RMS direction, then casts it into the BF16 residual hook. Write \(e^{\mathrm{req}}\) for the intended vector and \(e^{\mathrm{real}}\) for the change actually present after the hook. If rounding were negligible, those two would nearly match.

We score that match with the same two geometric ideas as linearity, now comparing request to realization at a single dose:

The full gate also checks both signed branches, their signed response (the half-difference defined in the design section), and the common-mode response. Fail any piece and that cell fails the fidelity check at that dose.

# Uses the same rms() / cosine helpers as the linearity sketch above.
def requested_realized_ok(
    e_req, e_real, *, cos_min=0.995, rel_rmse_max=0.10
) -> bool:
    return (
        cosine(e_req, e_real) >= cos_min
        and rms(e_real - e_req) / rms(e_req) <= rel_rmse_max
    )

If a tiny requested dose mostly rounds away, a “nonlinear” downstream curve could be noise from a mangled source edit; the fidelity check asks whether the source edit is faithful enough to blame the model for what happens next.

What we observed

At very small magnitudes, that rounding is not negligible. In this exact model, hook, prompt panel, and set of generic directions, the three smallest dose levels did not meet the predefined fidelity bounds. All 24 cells failed the full paired-branch rule at 0.5% and 1%, and 18 of 24 did so at 1.5%. Every cell passed from 2% through 30%.

At 2%, requested-versus-realized cosine had a median of 0.997, while median relative RMSE was 0.076. Passing does not mean requested and realized edits were bit-identical. It does show that the final-state nonlinearity on the 2%/3%/4% panel cannot be blamed on the low-dose fidelity failure seen below 2%. The boundary is consistent with BF16 rounding, but the experiment does not isolate rounding from other implementation details such as the casting path, kernel ordering, or fused operations. It is an empirical property of this implementation and panel, not a universal BF16 limit; the datatype and hook-path ablation under Next experiments is the direct mechanism test.

Source: this research note.

Figure 2: low-dose requested-versus-realized fidelity under this BF16 setup (file stem fig-3-bf16-delivery-floor; “delivery” is the protocol nickname). The upper curves show the central signed estimate; the bottom strip applies the complete rule to both signed branches, their signed response, and their common-mode response. Pale envelopes are the observed minimum and maximum across the 24 prespecified cells, not confidence intervals. Provenance: receipt.

Watch the Wake Develop Across Depth

At each state, gain is the root-mean-square magnitude of the signed response, \((h_{+}-h_{-})/2\), divided by the RMS magnitude of the realized signed source edit. It is 1x at state 50 by construction. A gain of 1.8x, for example, means the downstream signed response is 80% larger in RMS than the edit that landed. Median gain generally accumulated through later blocks, although individual trajectories were not uniformly monotone. Every tested trajectory ended above 1x.

The median final-state gain was 1.82x at 2%, 1.66x at 3%, 1.60x at 4%, 1.55x at 8%, and 1.48x at 30%. Relative amplification was generally larger at lower eligible doses (doses where every cell passed the requested-versus-realized checks), though the complete half-point dose series was not perfectly monotone.

Figure 3 makes the depth and dose axes explicit. The upper heatmap runs dose horizontally and post-block state vertically; the lower panel shows final-state gain across dose for every one of the 24 cells, with their median emphasized.

Source: this research note.

Figure 3: the measured hidden-state response from the intervention point to the final block (file stem fig-2-dose-depth-arc). The heatmap uses the median across the same 24 prespecified cells at each dose and layer; the lower panel retains every trajectory. These are descriptive summaries of exactly these 24 cells, not population estimates. Provenance: receipt.

Did the Jacobian Lens Predict the Wake?

Cross-dose curvature and predictive usefulness at one dose are logically independent: a response can bend as dose changes while a first-order map still captures useful structure at a fixed dose. Figure 1 used J only as a linear algebra reference. Predictive accuracy requires a different test: compare J’s prediction with the final response and ask whether it beats cheap alternatives. We used two: identity, which carries the observed source-state change forward unchanged (a geometry-free transport baseline, not a vocabulary readout such as a logit lens or tuned lens (Belrose et al., 2023)), and five prespecified random-J controls, which apply seeded sign and coordinate permutations to the released map.

The layer-50 source state was selected in advance for the primary comparison. On this fixed 3% panel, the released J cleared the absolute thresholds and decisively beat the strongest of five random maps in both metrics. For fixed-token logit correlation (agreement between predicted and observed score changes for 2,048 prespecified vocabulary tokens), real J scored 0.344; its advantage over the strongest random map was 0.286, with a 95% fixed-panel stability lower bound of 0.271. In residual space, the corresponding advantage was 0.337, with a lower bound of 0.321.

Identity was the harder baseline. J’s logit advantage over identity was only 0.011, with a lower bound of -0.0015, below the predefined 0.02 margin. Its residual-space advantage was slightly negative (-0.0015). No tested source-state layer cleared the added-value-over-identity margin in both spaces. The bounded conclusion is therefore precise: the released map contains structure beyond these five randomizations, but it did not show added value over simply carrying the residual change forward unchanged.

This comparison cannot separate two explanations for that null: that first-order transport itself adds little for these directions, or that corpus averaging washes out prompt-local structure that a per-prompt Jacobian would capture. A prompt-specific Jacobian baseline on the same cells is the control that would (see Next experiments); it was not part of this study.

For Figure 4, each horizontal position has a different source-state layer but the same target. At source layer \(\ell\), both \(J_\ell\) and identity receive the observed signed residual change after block \(\ell\) and predict the post-block-79 change. Only layer 50 is the intervention site. Layers 51–78 are shorter-horizon diagnostics after the model has already transformed the edit.

Source: this research note.

Figure 4: prediction from successive source states to one fixed post-block-79 target. Layer 50 was selected in advance for the primary comparison; layers 51–78 are descriptive shorter-horizon diagnostics, not additional intervention sites. Bands are fixed-panel stability intervals from resampling the eight prompts and do not support population-level inference. Provenance: receipt.

Interpretation

The result is sharp and bounded. Below 2% of residual RMS, requested edits did not land faithfully under BF16 casting in this setup, so nothing downstream of them can be scored; from 2% through 30% they did. On the 2%/3%/4% panel the realized edit stayed dose-linear, its fixed-J projection stayed within the same bounds, and the model’s actual final state fell outside both bounds in every cell: linear edit in, nonlinear hidden-state wake out, where “nonlinear” means the measured failure of the frozen dose-proportionality gates under BF16 execution, not an established smooth-dynamics mechanism. Along the way, median per-unit-edit gain generally accumulated through the remaining blocks, every trajectory ended above 1x, and amplification was largest at the smallest eligible doses. The corpus-average Jacobian sees nonrandom structure in that wake (it beat five randomized maps) but did not clear the added-value-over-identity rule at any tested source-state layer, so it does not outperform the simplest strong baseline: carrying the residual change forward unchanged. The depth arc localizes where the first-order description stops being adequate; it does not identify the mechanism that bends the trajectory.

Limitations

• Scope. Eight prompts, three generic Gaussian directions, one model (Llama 3.3 70B Instruct), one intervention site (after block 50), one token position, 24 cells. Nothing here estimates how often the pattern occurs across other models, layers, or directions, and nothing transfers to SAE decoder directions without the extension listed under Next experiments.
• Forward passes only. No tokens were generated, so no behavioral, SAE, deception, or consciousness-related outcome was measured.
• Mechanism unidentified. The gates record a departure from dose proportionality; they do not say what bends the trajectory. The candidate mechanisms in the Discussion are hypotheses.
• Precision not isolated. The 2% fidelity boundary is consistent with BF16 rounding but is not separated from the casting path, kernel ordering, or fused operations; it is a property of this implementation and panel, not a universal BF16 limit. Likewise, downstream finite-precision effects are not separated from higher-order model dynamics.
• Two readings of the identity null. First-order transport may add little for these directions, or corpus averaging may wash out prompt-local structure that a per-prompt Jacobian would capture. The design cannot tell them apart.
• Read off after outcomes. The threshold-sensitivity ranges in “Why these two gates” were computed after the data were opened. The layer 51–78 profiles in Figure 4 and the gain-by-dose series were prespecified as secondary, descriptive outputs of the frozen census (every post-block state and the per-row RMS gain were recorded by design) but carry no frozen pass/fail rule; the panel, gates, layer-50 comparison, controls, and 0.02 margin are the only prespecified decision rules.
• Fixed-panel uncertainty. Census ranges and the Figure 4 bands describe these 24 cells and eight prompts; they are not population estimates.

Two working practices bear on this result and were checked here. Using a corpus-average Jacobian as a transport predictor for generic residual edits did not show added value over the identity baseline by the frozen margin, and calibrating a steering dose on the assumption that the final-state response scales in proportion to the dose did not hold on the tested panel. Both statements concern three generic directions on one model; neither is a general prohibition.

Discussion: A Taylor-Series View of the Result

Corrected 4 September 2026; see the Corrections entry in Appendix D.

This section steps back from the audited numbers and asks what kind of object a Jacobian lens is, and why the observed split (every source edit linear, every fixed-J projection linear, every actual final state bent) is the pattern a linearization story predicts. Everything here is interpretation; the census results stand on their own.

A local Jacobian supplies the first-order term. Write \(F\) for the function taking the post-block-50 residual to the post-block-79 residual. For an edit \(e\) around the clean state \(h\),

where \(J(h)\) is the local Jacobian and \(H(h)\) the second-derivative term. The experiment records the signed branches and centers their contrast as \((F(h+e)-F(h-e))/2\). A prompt-local Jacobian would predict a response proportional to dose. The released lens instead averages Jacobians across other prompts and positions. The signed responses fail the dose-linearity gates at 2%–4% RMS; that establishes a departure from proportionality in this implementation, without isolating smooth higher-order dynamics from finite-precision effects.

Two sources of approximation remain unresolved. First, truncation: the lens omits every term past the first. Second, expansion point: it is not even \(J(h)\) for the prompt under study but a Jacobian averaged over other prompts and positions. The Figure 4 null (J failing to beat identity) could come from either. The prompt-specific Jacobian baseline under Next experiments is the control that separates them, and there is a reasonable chance that fixing the expansion point buys more than adding a derivative order.

Identity is not a dumb baseline; it is the free part of the first-order term. Because of the residual stream, the true Jacobian has the form \(I + \text{(block contributions)}\): skip connections carry the edit forward unchanged at no cost. Identity is that free part alone. The finding that the corpus-average J could not beat it says the learned, averaged remainder of the first-order term added nothing detectable for these generic directions.

Would a second-order lens be superior? In principle yes; as a shipped artifact, no. The first-order lens is an 8192 × 8192 matrix: about 67 million numbers. The second-order term is an 8192 × 8192 × 8192 tensor: roughly \(5.5\times 10^{11}\) numbers, on the order of a terabyte per source layer in BF16. What is feasible: Hessian-vector products computed on demand by double backpropagation, which give \(e^{\top}H\,e\) for a specific edit without materializing \(H\), and low-order polynomial fits along a fixed direction, since the whole dose curve is then a function of one scalar.

Falling gain does not identify the correction. Median final-state gain fell from 1.82x at 2% to 1.48x at 30%. This is compatible with saturation and with a negative higher-order correction. For example, \(F(e)=ae-be^3\), with \(a,b>0\), has signed response per unit dose \(a-be^2\), which decreases over the range where it remains positive. The gain curve alone therefore does not choose between a polynomial correction and a saturating response; the held-out-dose fits under Next experiments would.

Why would smaller doses be amplified more? At first glance the gain ordering looks backwards. It is not: the absolute wake still grows with dose (a 30% edit moves the final state far more than a 2% edit); what falls is the amplification per unit of delivered edit. The roughly 1.8x value is the gain at the smallest dose that passed the requested-versus-realized fidelity rule. It is not an established local-Jacobian gain: the 2%–4% panel failed dose-linearity, and the smaller doses failed the fidelity rule. Several mechanisms could contribute to the lower gain at larger doses. First, normalization compression: Llama applies RMSNorm on the way into every block, dividing the residual by its own overall scale. A tiny edit leaves that scale essentially untouched and rides through; a 30% edit measurably inflates the very norm it is divided by, so the network re-scales it downward everywhere downstream. Second, attention saturation: small perturbations move pre-softmax scores within their locally linear range, while large ones push attention patterns toward saturated, nearly winner-take-all configurations whose marginal sensitivity is lower. Third, trained robustness: a model trained on enormous data has seen its own representational noise and may have learned broadly contractive dynamics around its typical activations, so the farther an edit pushes the state off that manifold, the harder the remaining blocks pull back per unit of push. All three are hypotheses, not measurements; the first is also the most testable (the norm-corrected gain entry under Next experiments). The fidelity gate bounds the discrepancy between requested and realized edit sizes from 2% through 30%, but it does not by itself prove that the choice of denominator has zero effect on the gain curve.

The census established where the first-order description stops being adequate for these directions on this model; the mechanism that bends the trajectory, and whether any candidate “superior lens” would capture it, is a question for a study whose test is frozen before its outcomes are inspected.

Next experiments

Every follow-up named in this note, listed once and ranked roughly by cost, cheapest first. None has been run; each would need its own frozen plan.

1. Held-out-dose fits on the existing census. Fit low-order directional models (for example \(ae-be^3\)) on one dose panel and check them on held-out doses, to distinguish a polynomial correction from saturation. No new forward passes.
2. Norm-corrected gain. Recompute gain after accounting for the measured layer-wise norm inflation, which also quantifies how much the choice of denominator moves the gain curve; then, as a new run, deliver the edit as a norm-preserving rotation instead of an addition and check whether the dose profile of gain flattens.
3. A one-prompt public notebook that exports the requested edit, realized edit, fixed-J projection, and actual final residual for one prompt, one direction, and one layer at each dose, so the study’s central decomposition can be audited in minutes rather than hours.
4. Datatype and hook-path ablation (FP32, FP16, alternative hook implementations) on the same cells, which would turn “consistent with BF16 rounding” into a mechanism test and separate finite-precision effects from model dynamics downstream of the edit.
5. A prompt-specific Jacobian on the same cells, the control that separates the two readings of the identity null: truncation versus expansion point.
6. The SAE-direction extension. The measurement design transfers; the dose boundary does not. Replace the generic vectors with target-SAE, matched-SAE, and norm-matched generic directions; rerun the requested-versus-realized fidelity and common-mode checks for every vector; treat the 2%/3%/4% panel only as a candidate calibration starting point; add the study’s own controls and a never-intervened counterfactual branch; and specify the target features, prompts, dose panel, semantic endpoints, and analysis rules before their outcomes are inspected. The generic-direction result motivates this experiment; it does not confirm an SAE mechanism. Update (October 2026): a later, separately frozen source-aligned study in the same repository captured 40 paired J-lens cases under the accepted six SAE coordinates in Llama 3.3 70B (native-BF16 execution, FP32 readout, two capture seeds), with matched-SAE panels, identity, and five random-J comparators. It checked requested-versus-realized delivery and paired readouts at layers 50, 65, and 78 at fixed signed doses (±0.7 for individual features, ±0.5 for the aggregates); it did not rerun this note’s cross-dose linearity gates. Both the Jacobian lens and identity transport showed a downstream intervention footprint, which that study reports as propagation, not mediation.

Reproducibility And Artifact Ledger

Compact map for readers who already know what they want. Sample records and the teaching inventory are in the appendix.

Table: compact ledger. Freeze and release commits are separated on purpose. The 2.23 GB raw residual tree stays off-repository; its inventory SHA-256 is in the appendix.

Technical Appendices

Appendix: release inventory

Table: what "complete" means for this release. Middle column is plain language; right columns carry specialist pins and access conditions.

Open a record: compact publication marker

Plain English. The public package seals that the audit finished from the unchanged raw tree and records the hashes a skeptic should match.

Technical. PUBLICATION_COMPLETE.json in the compact replication record: status = complete_atomic_audit_only_recovery; summary_file_sha256 = b490b101…08f6; audit_receipt_sha256 = 176e2f7e…ee1; run id signed-dose-a084caa-wl8obvtuq0ax8t-v2.

Open a record: linearity census receipt

Plain English. Figure 1’s numbers are not typed by hand. A receipt lists every plotted range and failure count.

Technical. fig-1-linearity-census.receipt.json: actual_final_failure_count = 24; realized_source_failure_count = 0; j_of_realized_failure_count = 0; source summary SHA-256 b490b101…08f6.

Appendix A: Exact Test

The experiment used Meta’s Llama 3.3 70B Instruct checkpoint at revision 6f6073b… and Neuronpedia’s Llama-specific Jacobian-lens release at revision a4114d7…. Neuronpedia generated the maps with Anthropic’s reference method, averaging derivatives over 125 completed WikiText prompts of at most 128 tokens. They were not estimated from this study’s prompts or outcomes.

The three stress-test directions were independent standard-normal vectors in the 8,192-dimensional residual space, normalized to unit RMS and fixed by predeclared seeds. There was no semantic or SAE-based selection, rejection for outcome behavior, or post-hoc sign flip. Dose ran from 0.5% to 30% of residual RMS in 0.5-percentage-point increments. The prompts, directions, primary dose panel, and decision rules were prespecified.

Source: this research note.

Diagram: the study decomposition (requested edit, realized edit, predicted wake, actual wake) and the two frozen questions asked of them. Conceptual schematic; the quoted pass counts are from the audited census in the table at the head of this note.

For prompt \(p\), direction \(v\), and magnitude \(b\), the two signed branches produce the realized signed source edit (the quantity the implementation calls the central contrast)

The midpoint relative to the clean branch was retained as a separate common-mode diagnostic. The two signed branches therefore form one paired measurement, and a linearity cell is one prompt × direction curve across dose, giving 24 cells rather than 48.

For a realized layer-50 edit \(e_b\), the fixed lens supplies

The released \(J_\ell\) is an 8192 × 8192 corpus- and position-averaged input-output Jacobian from the post-block-\(\ell\) residual to the post-block-79 residual, with no prompt-specific conditioning, intercept, or centering term. The row-vector implementation is \(e_bJ_{50}^{\mathsf T}\). The dose-normalized statistics \(c_{\min}\) and \(d_{\max}\), their thresholds, and the requested-versus-realized fidelity criteria are defined once, in How we scored pass versus fail and the fidelity section; “slope discrepancy” is the implementation’s name for \(d_{\max}\).

Source: this research note.

Diagram: the two failure modes the frozen gates test, drawn as dose-normalized vectors \(q_b\). Conceptual schematic: arrow geometry is illustrative; the annotated ranges are the audited census values in the table at the head of this note and in Figure 1.

Figure 4’s bands are 20,000-replicate prompt-resampling stability intervals (repeated reweightings of the same eight prompts to show sensitivity within this panel). They are not confidence intervals for a broader prompt population. Each Figure 4 map takes a different post-block source state \(\ell=50,\ldots,78\) to the same post-block-79 target. Only block 50 is the intervention site.

Appendix B: Pilot Study and Motivation for the Signed-Dose Scan

An earlier target-masked pilot established that the released maps were loaded and oriented correctly and that the clean readout could distinguish its prespecified semantic and Yes/No controls. It did not meet the prespecified generic-intervention criterion, so it did not support a claim that J outperformed identity. More importantly for the present design, that pilot did not preserve the edit that BF16 arithmetic actually realized. Requested-versus-realized error and downstream curvature therefore remained confounded.

The signed-dose scan was built to resolve that one ambiguity by retaining the requested edit, realized edit, J reference, and actual state separately at every dose. It does not retroactively change the pilot’s threshold result. The full gate table, statistics, hashes, and execution history remain in the pilot replication record.

Appendix D: Audit, Provenance, and Corrections

The replication record identifies the study as consciousness_sae_signed_dose_scan_v1 and the model run as signed-dose-a084caa-wl8obvtuq0ax8t-v2. Its independent audit status is pass; the audit was recomputed from the unchanged raw-data tree without new model inference. The 2.23 GB raw dataset remains off-repository. Its complete file-inventory SHA-256 is 9fc8c8ffe5f8d34f8ddba863c11fff7370ef3644c1fad0139f96e39a4c8fbfc0, and the audited-summary SHA-256 is b490b101c112f774ae7bffc9c54294a70b91c874c4cee78eddaa7890446c08f6. The compact replication record contains the protocol pointers, result summary, audit receipts, and, as of commit f5e906e, the committed figure package.

All four empirical figures were generated programmatically from that audited summary with Matplotlib. The generator validates source identity, row counts, and zero-target guards; emits SVG, PDF, and 300-DPI PNG; and writes a separate JSON receipt containing data selection, transformations, derived values, alt text, and output hashes. Verification regenerated all 12 images byte-for-byte and matched the article’s alt text to the receipts; to repeat it, with the on-request audited summary in place, run python3 plot_signed_dose_scan_results.py --summary CALIBRATION_SUMMARY.json --output-dir . --verify --post index.md. This post bundle carries a publication-sanitized copy of the upstream generator, with machine-local receipt paths replaced by stable basenames: plot_signed_dose_scan_results.py (bundled SHA-256 0e9db35a7f1200d450b179ab38ecd9ce512bb95107cf613909ee2b71cc3ed39f; upstream pre-sanitization SHA-256 480c4b9ec2d9ea464119a9336053f5bb18838049274d7c623687f282047c25aa). It was not yet present at result commit fde24e9; that named gap was closed by committing the script, the four figure sets (SVG/PDF/PNG), the per-figure receipts, and the receipts index into the compact replication record at f5e906e, after re-verifying every file hash against the receipt provenance blocks and figure-receipts-index.json. The experiment repo at f5e906e or later can therefore be treated as a complete figure provenance root.

AI-assisted editorial review improved exposition but was not treated as scientific verification; neither pass received raw tensors or row-level data.

Corrections.

• 4 September 2026. Falling per-unit-dose gain does not rule out a negative cubic correction. The Taylor discussion distinguishes that possibility from saturation and no longer calls the smallest eligible dose an established infinitesimal linear regime. The census paragraph in What the census found distinguishes approximate input proportionality from the exact identity preserved by a linear map. The measured census, gates, and released receipts are unchanged; no model experiment was rerun for this correction.
• 2 October 2026. Revision: this edit reorganized the presentation (census table first, limitations and follow-ups collected in one place each, appendices trimmed) and corrected wording; no result, number, receipt, or figure changed.

Figure resources. Figure 1 receipt · PDF · PNG
Figure 2 receipt · PDF · PNG
Figure 3 receipt · PDF · PNG
Figure 4 receipt · PDF · PNG

References

• Gurnee et al. (2026), Verbalizable Representations Form a Global Workspace in Language Models, Transformer Circuits Thread, published 6 July 2026; the Jacobian-lens construction is in Methods.
• Anthropic, Jacobian-lens reference implementation, pinned commit 581d398613e5602a5af361e1c34d3a92ea82ba8e (Apache License 2.0).
• Neuronpedia, Llama 3.3 70B Jacobian-lens configuration, pinned revision a4114d7752d11eb546e6cf372213d7e75526d3a1.
• Meta, Llama 3.3 70B Instruct model card, pinned revision 6f6073b423013f6a7d4d9f39144961bfbfbc386b (Llama Community License; gated weights).
• Belrose et al. (2023), Eliciting Latent Predictions from Transformers with the Tuned Lens, arXiv:2303.08112; accessed 18 July 2026.
• Hernandez et al. (2024), Linearity of Relation Decoding in Transformer Language Models, ICLR 2024; arXiv:2308.09124; accessed 18 July 2026.
• Rimsky et al. (2024), Steering Llama 2 via Contrastive Activation Addition, ACL 2024, doi:10.18653/v1/2024.acl-long.828; accessed 18 July 2026.
• Taimeskhanov, Vaiter, and Garreau (2026), Towards Understanding Steering Strength, arXiv:2602.02712v2, 8 July 2026; accepted at ICML 2026; accessed 17 July 2026.
• O’Brien et al. (2025), Steering Language Model Refusal with Sparse Autoencoders, ICML 2025 Actionable Interpretability Workshop; arXiv:2411.11296v2, 22 May 2025; accessed 17 July 2026.
• Jones, T. (2026), Can a Jacobian Lens Detect SAE Steering?, Praxagent Research Note.