Technical Context
I dove into Redwood’s analysis and the paper itself because this isn’t a toy problem: if models can compute inside junk tokens, AI automation can no longer be judged solely by visible output. And yes, this isn’t just a neat metaphor—it’s observable behavior in newer models.
The core idea is simple. A prompt is padded with meaningless tail: dots, symbols, counters, empty tokens. Older models mostly ignored these, but fresh frontier models, as Redwood and the "Reading Between the Dots" paper show, use these regions as working memory for hidden intermediate computations.
What grabbed me wasn’t the term filler itself, but the mechanism. The model doesn’t have to output chain-of-thought externally, yet it can still push useful representations through hidden states at those positions. The authors demonstrate that this computation is partially decodable from internal states: early layers extract facts, later layers combine them.
The numbers aren’t empty either. Opus 4.5’s math accuracy without CoT jumped from ~45% to 51% with filler tokens; DeepSeek V3 saw even bigger gains on specific tasks. But an important caveat: this is more about parallel computational depth than long sequential step-by-step reasoning.
So filler tokens don’t turn the transformer into an infinite thinking tape. Rather, they give the model extra space to spread out sub-tasks, redistribute activations, and then assemble an answer. For interpretability, this is both elegant and unsettling.
Impact on Business and Automation
For applied AI implementation, my take is blunt: visible reasoning text is not equal to real reasoning. If you’re building checks, guardrails, or audits around “show your thinking,” that pillar just got weaker.
The second effect is architectural. Prompt engineering and AI integration now depend not only on perfect instructions but also on how context gives the model room for internal computation. This can boost quality on hard tasks while simultaneously breaking predictability and inference cost.
Who wins? Teams that test models at the behavior and trace level, not those who trust a pretty CoT. Who loses? Those selling “transparency” based just on output text. At Nahornyi AI Lab, we dissect exactly these issues in practice: where hidden computation helps and where it creates risk in AI solutions for business.
If AI automation already runs your critical processes, I wouldn’t wait for the model’s next surprise. Let’s calmly walk through your pipeline, check where control is illusory, and at Nahornyi AI Lab build AI solution development so the system saves time instead of introducing hidden risks at the worst moment.