5 Dimensions Out of 768 Control Everything

Ever since my control engineering studies, there's a principle that follows me everywhere: every complex system has control knobs, parameters you can turn to change behavior without breaking things.

In a rocket, you gimbal the engine a fraction of a degree and the whole trajectory shifts. In a drone, you tweak one rotor's speed and it rolls. In an EV, you adjust torque distribution and it handles differently.

And in an LLM? Until recently, I would have said the prompt. Maybe fine-tuning if you want precision.

Turns out it's more nuanced than that.

Let's imagine, you need a safety guardrail on your model. Instead of stacking system prompts that eat your context window, or fine-tuning a separate "safe" model that loses capability, you dial up the model's own internal safety dimensions. Instant. Reversible. No extra tokens, no GPU bill, no capability tradeoff.

Activation steering already exists: you compute a vector from contrasting prompts and inject it across all dimensions. It works, but it's blunt. Like turning every fader on a mixing board at once.

What this paper proposes is different. Out of 768 dimensions, a handful have magnitudes hundreds of times larger than the rest. Everyone assumed it was noise. The researchers' hypothesis: these are actually specialized semantic detectors, each tied to a specific domain. Their technique, Critical Dimension Steering, targets only these few dimensions instead of all of them. Like finding the 5 faders that control 80% of the sound and only touching those.

On their benchmarks, it outperforms conventional steering on both domain adaptation and jailbreak resistance.

I was curious enough to test it. Built a small guardrail demo that uses these critical dimensions instead of bolting on external filters. 92.5% accuracy, no retraining, a few lines of code. On a small model, sure, but I wasn't expecting it to work this well.

The same phenomenon shows up in Mixture-of-Experts architectures. In models like Qwen3-30B, DeepSeek-R1, and Mixtral, less than 0.5% of experts turn out to be critical: remove them and math accuracy drops 58%. Remove a few from DeepSeek-R1 and reasoning collapses entirely.

Whether the system is dense or distributed, you find these disproportionate leverage points. What we thought were anomalies might actually be the feature.

Today we have two proven levers to adapt an LLM:

→ Prompting: fast but coarse, costs you tokens every call → Fine-tuning: precise but expensive, slow, and you're stuck with it

Activation steering could be the third. And targeting only the critical dimensions instead of everything makes it both more precise and more interpretable. Not in production yet, but the direction is worth watching.