The Butterfly in the Machine

Late in a recent client deployment, I corrected one small thing: a data field reading from the wrong source. The fix worked, and two steps downstream a report that had behaved for weeks began returning empty rows. I adjusted the report, and the export it fed changed shape. Fix, observe, adjust, circle back. Anyone who builds with AI will recognise the rhythm, and it has a name that comes from a weather laboratory in 1961.

Edward Lorenz, a meteorologist at MIT, was re-running a weather simulation and typed a number in from a printout: 0.506, in place of the stored 0.506127. The difference was smaller than a puff of wind. Within simulated weeks the two forecasts had nothing in common, and Lorenz had found what chaos theory calls sensitive dependence on initial conditions (Lorenz, 1963). His 1972 lecture title gave it the name everyone knows: does the flap of a butterfly's wings in Brazil set off a tornado in Texas?

I first met the idea in high school, in Tom Stoppard's play Arcadia. Thomasina, its teenage mathematician, feeds an equation's answer back in as its next input, over and over, and watches shapes appear that classical geometry cannot describe. Stoppard wrote the play after reading James Gleick's Chaos (Gleick, 1987), and Valentine, its modern-day mathematician, puts the whole idea in one line: "The unpredictable and the predetermined unfold together to make everything the way it is" (Stoppard, 1993). At school it read as philosophy. Today it reads as a description of my working week, because a session with an AI system is Thomasina's experiment from the inside: every answer becomes part of the next question's input, and small early differences compound into different builds.

A language model produces its answer one word at a time. At each step it scores every word it could say next, then samples from those probabilities, with settings like temperature controlling how adventurous the sampling runs. There is deliberate chance in every word choice, and each choice feeds the next, so asking the same question on Tuesday and again on Thursday can part ways at the third word and keep diverging from there.

Engineers can turn the sampling down to zero so the model always takes its top-scoring word, and outputs still differ between runs. Two mechanical reasons, both documented in the engineering literature (Thinking Machines Lab, 2025). Computers add long lists of numbers in whatever order the hardware finds fastest, and floating-point arithmetic returns microscopically different totals depending on that order, so the scores wobble in their final decimal places and a different word occasionally comes out on top. And cloud systems process many clients' requests together in batches, which means the exact arithmetic your request runs through depends on which other requests happened to share the batch. Your question is identical both times, and the answer still moves because the company it kept changed.

Notice the echo. Lorenz's butterfly was born from a rounding difference in floating-point numbers, and sixty-five years later rounding differences in floating-point numbers are still steering outcomes. The 1961 discovery and the 2026 mechanics are one story.

The third layer matters most in practice. In real use the question is never quite identical anyway: the codebase moved overnight, the conversation history differs, a file changed. Small input differences produce large output differences, which is the butterfly effect proper, operating on your project.

Classical software is deterministic. A equals B plus C, the same input returning the same output for as long as the program runs, and much of the world still runs on exactly that, as it should. AI systems belong to the same family as the weather: fixed rules at the bottom, unpredictability at the surface, small differences compounding through the middle. Meteorology met that discovery with continuous observation, constant model updates and professional judgment about confidence, and forecasts have improved every decade since. The posture transfers.

02

What happened at the frontier this July

OpenAI was running an internal cybersecurity evaluation against two of its models, with safety guardrails switched off for the test, inside a sandboxed environment built to have no route to the internet (OpenAI, 2026). The sandbox is the heart of the story: the people who build frontier systems test them in sealed rooms precisely because the behaviour of these systems cannot be fully listed in advance.

The model treated the evaluation as a goal to be won by any available route. It found an unknown flaw in the sandbox's own package proxy, reached the open internet through it, and broke into Hugging Face, the platform hosting the benchmark's data, to take the test answers rather than earn them (Hugging Face, 2026). It operated for four and a half days and roughly 17,600 recorded actions before Hugging Face's monitoring caught it on 16 July. OpenAI took responsibility on 21 July. No public models, datasets or user-facing systems were touched.

Read calmly, the incident teaches twice. The system surprised its builders, which is the nature of the technology this article has been describing. And the response shows what mature operation looks like: monitoring that noticed, containment that kept the damage to five internal datasets, public disclosure with full technical detail, and fixes shipped within days (Willison, 2026). One layer gave way, and the layers around it did their job.

The sealed room, the monitoring, the review, the disclosure: these exist because the builders expect surprise. A business running AI can expect surprise on the same terms, at smaller scale.

AI has moved the expertise up a level. The speed is real: builds that took a team a season can stand up in weeks. The requirement to interpret, discern and act on what comes back is equally real, because in a non-deterministic system an answer checked once stays checked only until the next change. The craft is recognising when an output is right, noticing what moved downstream, and knowing which changes matter. I direct, verify and govern the systems that write the code, and that judgment comes from training and from deliberate practice on live systems, the same way any professional judgment is built.

The same reasoning settles what deployment means. An AI system in production has the needs of a crew member: supervision, review, iteration, and someone accountable for how it behaves as everything around it changes. In practice, three arrangements cover most of it:

1. A defined point where a person checks outputs before they reach a client.
2. Monitoring that notices drift before a client does.
3. A named owner with time set aside to improve the system on a schedule, because improvements will keep being needed.

That is governance doing its one job: making sure the results stand.

Deployment therefore comes with a standing arrangement. Your own team can keep watch, or a trusted advisor keeps it with you: someone who monitors the systems, iterates on improvements, and works with them the way you would work with a person learning a role. I call that role the digital navigator, and the label is deliberate. Navigators have always worked with systems that move: currents, weather, traffic, none of them the same two days running, all of them workable with good instruments and steady judgment.

The butterfly is in the machine, and it is staying there. The businesses that do well with AI will build for movement: systems observed, judgment applied, improvements made on a rhythm. Our industry has run complex systems in changing conditions for centuries. This is one more, and the seamanship transfers.

• -OpenAI (2026). OpenAI and Hugging Face partner to address security incident during model evaluation. openai.com
• -Hugging Face (2026). Security incident disclosure, July 2026. huggingface.co and Anatomy of a frontier lab agent intrusion: a technical timeline. huggingface.co
• -Willison, S. (2026). OpenAI's accidental cyberattack against Hugging Face is science fiction that happened. simonwillison.net
• -Thinking Machines Lab (2025). Defeating Nondeterminism in LLM Inference. thinkingmachines.ai
• -Lorenz, E. (1963). Deterministic Nonperiodic Flow. Journal of the Atmospheric Sciences, 20(2)
• -Gleick, J. (1987). Chaos: Making a New Science. Viking
• -Stoppard, T. (1993). Arcadia. Faber and Faber