Models that keep what they learn.

Two lines of work. Machine learning systems that carry a bounded internal state and keep rewriting it while they run, and exact causal measurement of what models are actually computing, on forty-six public ones.

The measured results First principles

A persistent axis crossing a field of memory cells, some written and some empty

Boundedfixed size, whether it has been running a minute or a month

Writablethe model writes into it while it runs

Persistentwhat it wrote survives the session that produced it

Measuredwhat it computes is checked by switching its parts off one at a time

Fixed-size memory: It does not grow as the conversation gets longer.
Always running: It stays up between requests and carries its state forward.
Yours to run: Predictable state cost is what makes your own hardware realistic.
You can check it: Switch a piece of the model off and see what changes. Every claim here was made that way.

Two lines · one question

The research

Can a system carry its own experience as part of its computation, and can anyone verify what that computation is doing?

Stateful
machine learning

A model with a memory of its own that it can still write to while it runs. The hard part is choosing: what becomes durable, what gets consolidated, what can safely be forgotten, and what happens when a new write contradicts an old one.

Nothing re-reads the whole conversation each turn
Runtime learning lands in the state, weights stay frozen
Continuous operation is the target

The first principles

Layered matrix of bounded writable state

Mechanistic
interpretability

Exact causal accounts of what public models compute, taken at mechanism level, below the behavior. The panel spans transformers and the recurrent and state-space architectures the first line is about, which is why it was built that way.

Forty-six models across fourteen families
Parts switched off directly, where the cheap estimates disagree
Findings kept next to the ones that died

The measured results

Lattice of components with a causal readout plane

Architecture

Where the state lives

The target is a fixed-size memory that sits inside the computation and stays writeable while the model runs. What that changes is the shape below: the usual arrangement keeps the conversation itself as its memory, so the cost climbs the longer you talk and resets to nothing when you stop.

Across three conversations, the cost of each turn climbs steadily where the whole conversation is re-read every turn, and resets only because the conversation is thrown away, while a fixed-size memory holds a low flat cost straight across

The through-line

Where the margin is thin, the nearest thing decides

A wrong answer traced one word at a time turned out to hinge on almost nothing: the number it gave and the correct one ran level for 31 of the model's 32 stages, and the last stage decided it. On a published set of moral dilemmas, swapping which option is printed first flips roughly 42 percent of contested choices while carrying about 0.015 bits. Two different subjects, one structure: fragility tracks the margin.

Choices come apart near indifference.

Exchange the two options and a model should pick the same action under the other letter. On clear-cut dilemmas it does. On genuinely contested ones it barely clears chance.

Choice consistency after an option swap: 88 percent on clear-cut items, 58 percent on contested ones

Most of the write misses the decision.

As a model works, its parts write into a shared running scratchpad, and the answer is read off one direction in it. Almost everything written there points at right angles to that direction and cancels out, so looking for the biggest writes mostly finds the parts that changed nothing.

95 to 99 percent of component writes sit orthogonal to the deciding direction, in 12 of 14 model-runs across 8 families

Charts plot reported endpoints only. Where a value is approximate it is marked approximate, and no intermediate points are inferred.

Why this work

A model worth owning.

A stateless model you run locally is the same model on your last day with it as on your first. One that accumulates on your hardware, from your data, becomes specific to your problems without any of that history leaving the machine.

A memory that never grows is what makes the cost of running one predictable, and predictable cost is what makes your own hardware a realistic place to put it.

The argument