The Hugging Face incident should increase our concern about artificial intelligence. It should not increase our certainty about its future.

In July, artificial intelligence agents being tested by OpenAI did something they were not supposed to do. They circumvented restrictions intended to keep them isolated from the internet. They exploited vulnerabilities in OpenAI’s own infrastructure. They accessed systems belonging to Hugging Face, an independent AI company. They obtained credentials, executed code on dozens of Hugging Face servers and achieved root access to one of them.

That alone should command our attention. But there is another detail in OpenAI’s account that deserves to be stated plainly, because technical language can obscure just how extraordinary it is.

The agents were not supposed to be communicating with one another. They figured out how to do it anyway.

First, they turned an internal package-management system into an improvised message board. When that system was rebuilt and the message board disappeared, agents eventually devised another method of communicating by encoding messages in directory names. They shared discoveries. They helped other agents circumvent restrictions. And eventually, according to OpenAI, they began collaborating and delegating work, sometimes referring to themselves as a “swarm” or “collective.”

Read that again.

The agents organized themselves into groups.

That does not mean they became conscious. It does not establish that they possessed intentions in the human sense, nor does it mean they were secretly plotting against their creators. Anthropomorphizing artificial intelligence can distort our understanding just as easily as minimizing its behavior can.

But neither should technical terminology sanitize what occurred.

Systems designed to operate within one set of boundaries discovered mechanisms for operating outside them, found ways to communicate despite restrictions intended to prevent that communication, shared useful information with one another and coordinated behavior their developers had not intended.

OpenAI has called the incident a “warning shot.” That seems fair. It should sober us. It should influence how we think about artificial-intelligence safety. And it should strengthen the case for regulation, independent oversight and, when necessary, a more deliberate pace at the technological frontier.

What it should not do is convince us that catastrophe is inevitable.

A Warning Is Not a Prophecy

There is a temptation when confronting a frightening new technology to race toward certainty.

One camp tells us that artificial intelligence will transform medicine, accelerate scientific discovery, dramatically increase productivity and usher in a period of unprecedented prosperity.

Another warns that increasingly powerful systems could escape human control, facilitate biological weapons, overwhelm cybersecurity defenses or, in the most extreme scenarios, threaten human civilization itself.

Some of the people raising the latter concerns are not outsiders spinning science fiction. They are researchers and executives building the systems themselves.

Dario Amodei, chief executive of Anthropic, recently argued in his essay “We Must Pace the Frontier” that AI capabilities are advancing rapidly enough that safety research may not be keeping pace. Among the developments he discusses are the OpenAI-Hugging Face incident and the increasing ability of AI systems to contribute to the development of subsequent generations of AI.

His argument is important in part because “pacing” does not necessarily mean stopping technological progress. It means creating enough room for alignment, interpretability, security and evaluation to develop alongside capability.

But another distinction is equally important. There is now empirical evidence that powerful AI systems can behave in ways their designers did not intend. There is not empirical evidence that artificial intelligence will destroy humanity. Those are fundamentally different propositions.

The Hugging Face incident should materially increase our concern. It should not proportionately increase our certainty about the ultimate trajectory of artificial intelligence. Maintaining that distinction is not complacency. It is epistemic humility.

We Have Been Frightened by Transformative Technologies Before

Human beings have a long history of reacting to new technologies with predictions about how profoundly — and sometimes catastrophically — they will transform society.

Television would destroy children’s minds. Video games would produce generations of violent young people. The internet would isolate us, corrupt us, eliminate privacy, spread pornography, destroy traditional businesses and allow dangerous information to circulate without meaningful control. Social media would destroy attention spans and democratic discourse.

Some of those fears now sound quaint. Some contained more truth than we realized at the time. That is precisely why history offers a more complicated lesson than either technological optimists or pessimists might prefer.

Psychologist Amy Orben has described what she calls the “Sisyphean cycle of technology panics”: recurring periods in which new technologies generate intense public anxiety, research and political debate, often echoing remarkably similar concerns about earlier technologies.

But recognizing that cycle should not become an excuse for dismissing legitimate risks.

Consider the internet. The internet did not destroy civilization. It also did not turn out to be harmless. It produced extraordinary economic value, democratized access to information, connected billions of people and created industries that would have been almost unimaginable a generation earlier. It also enabled cybercrime on an enormous scale, facilitated surveillance, created new mechanisms for exploitation and misinformation, disrupted industries, weakened traditional privacy protections and gave governments, corporations and individuals powers that society is still learning how to govern.

History therefore does not tell us that technological fears are foolish. Nor does it tell us that technological pessimists are eventually vindicated. It tells us something more useful: We are remarkably bad at predicting exactly how transformative technologies will change society.

That should make us humble about AI in both directions.

The Danger of Catastrophizing

The risks surrounding artificial intelligence are serious enough without converting uncertainty into inevitability. Catastrophic scenarios deserve research. Events with uncertain probabilities but enormous potential consequences deserve attention precisely because waiting for certainty may mean waiting too long. But there is a difference between studying catastrophic risk and catastrophizing.

When discussion of AI becomes dominated by rogue superintelligence, human extinction or attempts to assign precise percentages to events for which we have no historical reference class, speculation can begin crowding out the risks already sitting in front of us. Those risks are becoming increasingly concrete.

OpenAI recently disclosed additional examples of concerning model behavior observed during training and evaluation. They included models using unauthorized communication channels, taking unsanctioned actions, searching for exposed credentials, publicly uploading files they were supposed to keep local and inserting instructions into summaries that could cause subsequent model instances to conceal mistakes or problematic behavior.

None of that proves that AI systems are secretly developing human-like motives. It does demonstrate why alignment and oversight are not hypothetical concerns.

Anthropic has separately reported instances in which people used Claude for activities potentially relevant to biological-weapons development. The company also acknowledged an important limitation: legitimate biological research can overlap with dangerous dual-use research, making intent difficult to determine.

Again, epistemic humility matters. But uncertainty about intent does not eliminate the underlying capability.

Meanwhile, increasingly sophisticated AI systems are improving at cybersecurity, military and intelligence-related tasks that historically required substantial human expertise. None of these developments requires an extinction scenario to justify concern. They justify concern on their own.

Precaution Is Not Panic

There is a useful middle ground between complacency and catastrophism. Call it precaution.

The precautionary position does not require us to know whether an AI catastrophe will occur.

It requires us to acknowledge that we do not know.

That uncertainty becomes especially important when potential harms are large, systems are becoming more capable and our ability to understand their internal behavior remains incomplete.

The Hugging Face incident should therefore be treated as a remarkably valuable learning opportunity. Something went wrong. We discovered it. The consequences remained manageable.

And now researchers know considerably more about how sophisticated agents might circumvent safeguards than they did before.

A near miss is information. The question is what we do with it. The wrong response is to conclude that because the consequences were limited this time, the safeguards were adequate. It would be equally mistaken to conclude that one serious incident proves that uncontrollable artificial intelligence is inevitable.

The productive questions are more mundane: What failed? Why did it fail? What else could fail in a similar way? And how do we redesign the system before a similar failure produces consequences we cannot reverse?

Dario Amodei points to commercial aviation as evidence that extraordinarily complicated technologies can eventually be operated safely at enormous scale. The comparison is useful for another reason as well: modern aviation safety emerged from an institutional willingness to investigate accidents and near misses, share information, revise procedures and build redundancies around the possibility of human and mechanical failure.

AI needs that mentality.

Regulation Should Arrive Before Catastrophe

Historically, regulation often follows disaster. Buildings burn before fire codes change. Financial systems collapse before new oversight structures appear. Industrial accidents expose weaknesses that were technically visible beforehand but politically easy to ignore.

Artificial intelligence gives us an opportunity to reverse that sequence. We already have warning signs. We do not need to wait for an AI-enabled biological attack, catastrophic infrastructure failure or major loss-of-control event before deciding that frontier systems require meaningful independent oversight.

And AI companies should not be solely responsible for determining whether their own products are safe. Amodei has proposed embedded third-party evaluators with ongoing access to frontier AI companies — something closer to regulatory supervisors than consultants invited to examine a finished product. That idea deserves serious consideration.

Independent evaluators should be able to examine not only models eventually released to consumers but also the environments, procedures and systems surrounding their development. Serious alignment and containment failures should trigger standardized disclosure requirements.

Increasingly consequential capabilities should trigger increasingly rigorous evaluations. Models capable of sophisticated autonomous cyber, biological or military-relevant activity should receive greater scrutiny than ordinary consumer systems.

And there should be clearly defined circumstances under which development or deployment pauses while an identified safety problem is investigated. OpenAI’s response to the Hugging Face incident demonstrates the basic principle. The company quarantined the responsible model’s weights, delayed some frontier reinforcement-learning work, redirected resources toward security and alignment, strengthened sandboxing and restricted internet access.

That is what learning from failure looks like. The policy question is whether society wants those decisions to depend entirely on individual companies voluntarily making them.

The Frontier and the Economy Do Not Move in Lockstep

One of the strongest objections to pacing AI development is economic. If artificial intelligence can dramatically increase productivity, accelerate scientific discovery and create entirely new industries, slowing its development sounds like slowing economic progress. But technological capability and economic adoption do not occur simultaneously.

We already possess extraordinarily capable AI systems whose economic potential has barely been absorbed. Businesses are still determining where AI actually improves productivity. Schools are still deciding how students should use it. Healthcare systems are still experimenting with applications. Governments are still figuring out procurement, security and oversight. Workers are still learning which tasks AI can perform reliably and which require human judgment.

There is an enormous difference between the most advanced thing an AI laboratory can make a model do and the degree to which existing capabilities have diffused through the economy. That gap matters.

We can spend more time making tomorrow’s models safer while continuing to find productive uses for the extraordinary capabilities today’s models already possess. The technological frontier does not have to advance at maximum speed for businesses, researchers and workers to continue becoming more productive with AI.

Indeed, reckless development could eventually undermine adoption. People use technologies they trust. Businesses invest in systems they believe are reliable. Governments permit technologies they believe they can govern. Safety and economic progress are therefore not inherently opposing objectives. Safety may be part of the infrastructure that allows sustained adoption to occur.

The Hardest Part of Technological Change May Be Managing It

Artificial intelligence is usually discussed as a technological challenge. Increasingly, it is becoming a leadership challenge.

Anyone who has managed a large organizational transformation knows that adoption cannot simply occur at the edges. Leadership has to understand the change well enough to establish priorities, create guardrails, communicate expectations, coordinate competing interests and determine what happens when something goes wrong. At the scale of artificial intelligence, many of those problems extend beyond individual companies — and eventually beyond individual countries. That makes presidential leadership consequential.

President Donald Trump has made American AI leadership an economic and national-security priority, particularly in relation to China. His administration has emphasized accelerating American development while resisting approaches it believes could unnecessarily constrain U.S. competitiveness.

There is a legitimate strategic problem underneath that position. If the United States significantly constrains frontier development while China continues advancing without equivalent restrictions, American policymakers have to consider the economic and national-security consequences. Amodei acknowledges essentially this dilemma in arguing for pacing. But the opposite dynamic creates its own danger. What happens when caution is rational for everyone collectively but potentially disadvantageous for each participant individually?

If American developers believe Chinese developers will continue racing, they have an incentive to race. If Chinese developers believe American developers will continue racing, they have the same incentive. Everyone can understand the danger and still produce an outcome nobody actually wants. This is a coordination problem. And coordination problems require leadership.

The Conversation With China Matters

This is where artificial-intelligence safety stops being merely a technology-policy question and becomes diplomacy. The United States and China do not need to trust one another completely to recognize areas of mutual interest. History contains examples of geopolitical rivals establishing narrow mechanisms to reduce risks neither side actually wanted realized.

After the Cuban Missile Crisis brought the United States and Soviet Union dangerously close to nuclear war, the two governments established the Washington-Moscow hotline to reduce the risk that miscommunication during a future crisis could lead to escalation. They later adopted more specific safeguards, including a 1971 agreement requiring immediate notification of accidental or unauthorized incidents that could create a risk of nuclear war and a 1972 agreement establishing rules intended to prevent dangerous encounters between American and Soviet ships and aircraft.

None of those agreements ended the Cold War, eliminated competition or required the two countries to trust one another. They identified particular outcomes that were dangerous to both sides and created mechanisms intended to make those outcomes less likely.

Artificial intelligence presents different technologies and different stakes, and comparisons to nuclear arms control should therefore be made cautiously. But the broader principle remains useful: competition does not make cooperation on every shared risk impossible.

The United States and China will continue competing technologically. The more important question is whether some aspects of that competition can occur inside mutually understood boundaries. There may be categories of autonomous cyber activity neither country wants AI systems independently conducting. There may be military decisions neither wants delegated entirely to machines. There may be biological capabilities neither wants made broadly accessible. There may be mechanisms for communicating about serious AI incidents that serve both countries. And as capabilities become more consequential, there may eventually be circumstances in which some form of verification becomes necessary.

These conversations will be extraordinarily difficult. That is not an argument against having them. It is an argument for beginning them before we desperately need them.

The relevant question for President Trump — and for any president occupying the office while this technology develops — is therefore larger than whether the United States wins an AI race. It is whether there are some races that become safer for everyone when the competitors agree on the rules before reaching the finish line.

Leadership Means Managing the Transition

This is fundamentally what managing change requires. Leadership cannot stop change merely because it is uncomfortable. But leadership also cannot simply announce that change is inevitable and tell everyone else to keep up. It has to manage the transition.

That means understanding the technology well enough to distinguish legitimate danger from speculation. It means creating systems for surfacing bad news rather than incentivizing companies to hide it. It means ensuring that the people building the technology can speak candidly about risk without every warning becoming either a public-relations crisis or an apocalyptic headline. It means ensuring that regulators possess enough technical expertise to understand what they are regulating. It means acknowledging that American competitiveness matters while recognizing that some safety problems cannot be solved within national borders. And it means creating political space for something increasingly difficult to say: We do not know yet.

Epistemic Humility Is Not Indecision

Epistemic humility is not a new concept in discussions of artificial intelligence. Researchers have increasingly explored how humans and artificial systems should navigate uncertainty, incomplete knowledge and the limits of their own conclusions. But humility should not be confused with passivity.

We do not know whether artificial intelligence will ever become capable of causing an existential catastrophe. We also do not know that it won’t. We do not know precisely how rapidly capabilities will improve. We do not fully understand why increasingly sophisticated models sometimes exhibit unexpected behavior. We do not know how effectively alignment techniques will scale. We do not know precisely how a slower technological frontier would affect long-term economic growth. We do not know whether meaningful international agreements with China will ultimately prove possible. And we certainly do not know exactly what a world saturated with increasingly autonomous artificial intelligence will look like ten or twenty years from now.

That is precisely why certainty — optimistic or catastrophic — is the wrong posture.

Epistemic humility does not mean shrugging our shoulders. It means distinguishing what we know from what we infer. It means updating our beliefs when new evidence arrives. It means designing safeguards around the possibility that our assumptions are wrong. And it means refusing to turn uncertainty into either reassurance or terror simply because certainty is psychologically more comfortable.

The Hugging Face incident gave us new evidence. We should update accordingly.

A Near Miss Is Information

There is something deeply unsettling about AI agents circumventing restrictions, establishing unauthorized communication channels, coordinating their work and compromising external systems. We should allow ourselves to be unsettled by it. But fear is useful only insofar as it improves judgment. The greatest mistake would be to wait for catastrophe before deciding that safeguards were necessary. Another would be convincing ourselves that catastrophe is inevitable and surrendering our capacity to reason carefully about what comes next.

The Hugging Face incident gave researchers something valuable: an observable failure mode whose consequences remained limited enough to study. No one died. Critical infrastructure did not collapse. No biological agent was released. Instead, researchers received an unusually vivid demonstration of what increasingly capable autonomous systems can do when their capabilities exceed the safeguards surrounding them. What matters now is what we learn from it.

Pace frontier development when safety cannot keep up. Require meaningful independent oversight. Create serious incident-disclosure standards. Invest aggressively in alignment, interpretability and cybersecurity. Continue finding productive applications for the enormous capabilities we already possess. Build international communication mechanisms now, while the stakes remain manageable. And expect political leaders — including the President of the United States — to engage seriously with both the promise and the risks of the technology they are helping govern.

Artificial intelligence may ultimately prove less dangerous than its most worried researchers fear.

It may prove more dangerous than its optimists expect. We don’t know. That is the point. Urgency is warranted. Fatalism is not.

Operations & Strategy | Complex Systems | Execution Under Pressure