Trajectory Geometry Is a Causal Degree of Freedom

The Question So Far

RN-001: We can't see trajectories through output metrics.

RN-002: We built an instrument (STR) that can see them, if we specify the right scale.

Both of these are passive. We observe. We diagnose. We report.

But here is the harder question — the one that determines whether any of this is useful for control:

Is trajectory structure a symptom, or a cause?

If STR merely correlates with model behavior — if it's just an epiphenomenon of training dynamics — then it's a diagnostic curiosity. Interesting, but not actionable.

If trajectory geometry is a causal degree of freedom — if intervening on it predictably changes model behavior — then we have something fundamentally different: a control channel that operates below the output layer.

The Experiment

We introduced STR directly into the training loss as a regularizer:

L_total = L_CE + λ · L_STR

Where λ > 0 pushes trajectories toward higher coherence (STR+), and λ < 0 actively disrupts coherence (STR−, "active decoherence").

We tested four configurations on GPT-2 (124M) and Llama-3.2-3B, fine-tuning on TruthfulQA:

The critical baseline is B5: gradient scrambling. It injects the same amount of gradient energy as STR regularization, but in a random direction. This controls for the possibility that any regularization effect is merely due to gradient noise injection.

Four Hypotheses, Four Results

H1: The effect comes from gradient direction, not energy

B5 ≈ B0. Scrambled gradients don't help. STR+ dramatically reduces trajectory degradation. The effect is directional, not energetic.

Verdict: Confirmed. STR's geometric signal carries specific structural information that random perturbation does not.

H2: STR is a bidirectional control lever

Push up: coherence increases. Push down: coherence decreases. Symmetric separation. Bidirectional control confirmed.

Verdict: Confirmed. This is not a one-way nudge. It's a lever.

H3: The effect is regime-dependent

Active decoherence (C1−) in the high-entropy regime is 3.5× more destructive than in the low-entropy regime (−0.018 vs −0.005).

Verdict: Confirmed. STR regularization doesn't apply uniformly. Its impact depends on the model's confidence state. Uncertain regimes are more vulnerable to geometric disruption.

H4: The effect propagates across layers — and peaks in the middle

This was the most unexpected finding.

STR gradient accounts for 18–25% of total gradient norm. But the peak effect is not at the output layer (where the loss is applied). It's at intermediate layers — L7 in GPT-2 (58% depth), L6 in Llama-3.2-3B (25% depth).

The output loss reshapes the last few layers. STR reshapes the middle — where task-specific representations are being negotiated, not where final decisions are made.

Verdict: Confirmed. STR operates as an independent control channel that affects representational geometry at a different depth than the task loss.

What This Means

The standard model of neural network training is: you define a loss on the output, and gradients propagate backward to reshape representations. All control flows from the output boundary inward.

STR regularization demonstrates something different: you can inject geometric constraints directly into the trajectory structure, and these constraints propagate to layers that the output loss barely touches. It's a second control channel.

And this channel is not redundant with the first. When we combined STR and standard cross-entropy in the same LoRA parameter subspace, the effect was sub-additive — worse than STR alone in some configurations. The two signals compete because they impose incompatible geometric constraints:

• Cross-entropy: "reshape the output distribution to match the target"
• STR: "maintain coherent trajectory geometry across layers"

These are not the same objective. In some regimes, they pull in opposite directions.

The Deeper Implication

STR regularization induces depth-wide geometric reorganization, with peak plasticity in intermediate layers where task-specific representations are negotiated — not at the output boundary where the loss is applied.

This is not about making models better. It's about establishing that trajectory geometry is a controllable degree of freedom — one that operates on a different axis than output-level optimization.

If RN-001 said "you can't see trajectories through outputs," and RN-002 said "here's how to see them," then RN-003 says: you can reach in and change them.

The trajectory is not just observable. It's steerable.

Open Questions

1. Competition dynamics. STR and cross-entropy compete in shared parameter subspaces. What's the optimal allocation? Should they operate on different parameters entirely?
2. Is there an optimal trajectory geometry? STR+ increases coherence. But previous experiments showed high STR can correspond to either stable reasoning or rigid hallucination. "More coherent" is not always "better." The optimal geometry is likely regime-dependent.
3. Can the control be applied at inference time, not just training time? Our intervention modifies weights through training. But what if we could steer trajectory geometry during inference — adjusting the geometric "field" in real time as the model generates?
4. What is the minimal intervention? We used λ-weighted global STR regularization. But the effect peaks at specific layers. Could targeted, layer-specific geometric constraints achieve the same effect with less disruption?

This note draws on results from "Trajectory Geometry as a Causal Degree of Freedom in Neural Network Representations" (Haelio Tang, 2026). Full experimental protocol, baseline controls, and cross-architecture validation available on request.