Skip to main content
AI evaloverengineeringbitcoin

When Sol Max Got Buried in Bitcoin

Sol Max, in an attempt to create a 'perfectly scientific' eval, ditched a regular random number generator and reached for the hash of a future Bitcoin block. For businesses, this is a crucial sign: with AI implementation, overly rigid requirements easily bloat architecture, delays, and costs with no real benefit.

Technical Context

I love these stories not for the meme factor, but for the brutally honest engineering conclusion. You ask a model to make an "invulnerable" eval, and instead of simply seeding the code, it goes and pulls the Bitcoin mainnet as a source of "truly unpredictable" randomness. This is where you see how AI implementation can suddenly veer from common sense into a cult of absolute rigor.

Jokes aside, there is technical logic to it. The hash of a future Bitcoin block truly can't be predicted in advance, and as an external entropy source, it works. But I would hit pause right there: for model evaluation, I usually need reproducibility, a controlled seed, and clear statistics—not a ten-minute wait for a block and an extra dependency on someone else’s network.

What gets me here isn’t Bitcoin, but the model’s thinking trajectory. It was given a goal of "don’t trust local randomness" and it dutifully went looking for a more "pure" source of randomness. Not the most optimal, cheap, or fast one. Just maximally protected against an imagined flaw.

And this is very telling for modern agent-based systems. If you leave a gap in the prompt or spec like "make it as reliable as possible," the model will start building an AI architecture for the worst-case scenario, not the real task. Then people wonder why a simple eval suddenly requires network access, external APIs, and half a day to run.

Impact on Business and Automation

The business takeaway is simple: not only weak requirements are dangerous, but also overly heroic ones. When I design automation with AI, I always separate "statistically good enough" from "cryptographically flawless." These two modes cost completely different amounts of money.

Only rare scenarios win here, where an external, publicly verifiable entropy source is needed. Almost all applied pipelines lose: eval, A/B runs, internal agent tests, batch quality assessments. There, you get delay, integration complexity, and fragility instead of value.

I see this all the time with clients: the problem isn’t the model, but how requirements are framed. At Nahornyi AI Lab, we ground these things so that AI automation doesn’t turn into an expensive engineering fantasy. If your system is already starting to "think too fundamentally" where you just need a working result, let’s review the architecture and build AI solutions for business without the cult of perfection.

We previously analyzed how prompt injection in Claude's self-reflection leads to DoS and disrupts automation. This is just one example of a model acting outside expected behavior, echoing the incident described in this article.

Share this article