How is time represented in LLMs?

Can we measure the time horizon behind a language model's output?

So far, a direction in the activations of two Qwen3 models can be read and steered, shifting outputs between short- and long-term options. In one of them, horizons from seconds to centuries form an ordered map. These results come mainly from choice tasks, and open-ended output has only a rough, unvalidated score so far.

Read the argument See every number
A scatter plot of model activations. Colored clusters run in order from seconds through minutes, hours, days, weeks, months, years and decades to centuries, with no-horizon samples in a separate gray cluster.
  • Seconds
  • Minutes
  • Hours
  • Days
  • Weeks
  • Months
  • Years
  • Decades
  • Centuries
  • No horizon
Time-horizon geometry in Qwen3-4B, projected with PCA. A projection shows structure, not a viewpoint inside the model. From Rios-Sialer et al., Figure 1.

How does a time horizon become an output?

The current answer, step by step, with how much weight each step can carry.

Where is it computed?

In Qwen3-4B, a subgraph across layers 17–35. Attention at L24 is flagged by all four non-probing methods, and MLP effects concentrate at L31–35.

Methods largely agree, one model

What form does the horizon take?

Ordered clusters from seconds to centuries, on a curved manifold.

Qualitative PCA evidence, one model

When does it become a preference signal?

The horizon direction is unstable across prompt positions until the user-to-assistant boundary, where attention collapses it into a binary signal.

Qualitative PCA evidence, one model

Does a stated deadline reach the output?

Not reliably. When only the sooner option can pay out by the deadline, Qwen3-4B outputs select the undeliverable long-term option 47–53% of the time.

960 prompts, compared across 30 models

Does changing the signal change the output?

Yes, in the settings tested. At L44 of Qwen3-32B, long-term outputs move from 33.7% at steering strength −128 to 76.8% unsteered to 93.1% at +128. In Qwen3-4B, steering at L22 gives about 3.4× odds.

Controlled in Qwen3-32B, suggestive in Qwen3-4B

Is it really about time?

Not established. Chain reliability is decodable at about 95% in the same subgraph, and the Qwen3-32B direction may also carry abstraction, urgency and framing.

Open

How far does this generalize?

Unknown beyond two Qwen3 models. A direction transferred to a monetary task in Qwen3-32B, but both papers test one model family in thinking-disabled settings.

Open

Reading a signal and changing it are different tests

In both studies, the layers with the best probe accuracy differ from the layers with the largest steering effect. A probe used for oversight has to track what affects the output, so accuracy alone isn't enough.

Qwen3-4B 36 layers

Best probe accuracy
Peak at L26, plateau from about L17
Top patched components
L21 and L24 attention, L31 and L35 MLPs
Largest steering effect
L19–22 strongest, weak at L26

Qwen3-32B 64 layers

Best probe accuracy
L24, L28 or L32, by probe family
Top patched components
Not measured in this study
Largest steering effect
Choice gap largest at L44, logprob gap at L48 by a small margin
Layers are placed by their fraction of each model's depth. Solid marks show the strongest layers; the tint shows the range swept or the plateau. The methods differ between studies, the Qwen3-32B sweep only covers L24 to L48 in steps of four, and its authors claim no uniquely optimal layer, so this comparison is suggestive.

What these results don't show

No surface baseline for the probes

Shuffled labels rule out chance, but no text-only baseline has been run on these datasets, and the same kind of baseline explained an apparent planning signal in generated code.

No prompting baseline for the steering

Asking for a short- or long-term answer might produce similar shifts. The strongest steering settings are also large perturbations.

One model family, narrow tasks

The two papers cover Qwen3 models with thinking disabled, mainly on choice tasks. Multi-turn and agentic settings are untested.

Not yet a monitor

No result here shows a probe flagging a change in advance. A related study of repetition looked for that lead time and found none.

Each gap is open work. See what you can pick up.

Papers