The internet could be on the verge of a total meltdown. Yesterday, someone posted to a public forum that two major security flaws had been found in a programming language that undergirds the web. The language, called PHP, is used by as much as 70 percent of websites.

In order to protect themselves from hackers, a number of government, hospital, and school websites across Europe went dark this morning. The organization that maintains PHP quickly confirmed it was preparing to announce two major vulnerabilities alongside patches for them. Both weaknesses, officially disclosed today, appear to have existed in PHP for more than a decade but were only just discovered with the use of AI tools. Hackers have already been detected trying to exploit them, likely with the help of AI, and implementing the fixes to address the vulnerabilities could take companies days or weeks. In the meantime, tens of millions of servers, databases, and devices are exposed. “PHP is one of the fundamental pieces of technology that supports the internet,” Sam Rubin, a senior vice president at the cybersecurity firm Palo Alto Networks, told me. As of now, the websites of the Finnish health agency, the Port of Rotterdam, the Paris police force, 4Chan, and an unknown number of others are all down while they address the PHP vulnerabilities.

[Read: How long until AI hacks everything?]

News of these weaknesses in PHP is the capstone in a weeks-long slew of devastating, AI-powered cybersecurity events. In recent weeks, criminals and state-backed attackers have used AI to hack into more than 100 companies, including John Deere and SharkNinja; to automate hacks on some 400 financial institutions in Latin America; and to steal hundreds of thousands of credit-card records from two companies. Since this summer, attackers have hit governments, data centers, and hospitals in Ukraine, Russia, Argentina, Bangladesh, and Indonesia. Also today, the developers of Node—a platform used by nearly half of software developers—disclosed multiple high-severity vulnerabilities. And that’s the abridged list.

For the better part of a year, cybersecurity experts have been warning, with mounting alarm, about an impending cyber-apocalypse. It may have begun. For decades, the internet has been riddled with holes and bugs. Code is finicky, and the standard IT approach has been to fix problems when they arise. For decades, the balance held: Skilled hackers are few, and finding and exploiting vulnerabilities takes time.

Recent advances in AI have totally upended that equilibrium. Top models have become very good at finding holes and writing malware, and can work much faster than even a team of humans. This is true of both open-source AI systems from China, which can be downloaded and used without restrictions, and the even more capable models from OpenAI and Anthropic, which are continually being jailbroken. The rate of software vulnerabilities identified per month has doubled over the past year. Meanwhile, the time that hackers need to begin exploiting those vulnerabilities—figuring out how to break in and out without being caught—has fallen, by some estimates, from weeks to hours. An actual intrusion, if automated with AI, can take as little as 39 seconds. Historical approaches to detecting and fixing vulnerabilities, or monitoring for suspicious internal activity, are insufficient.

AI has created new vulnerabilities, but it also promises to fix them: The vulnerabilities in PHP were discovered independently by a Google-owned cybersecurity company and another effort led by OpenAI. Google, OpenAI, and Anthropic, along with other tech firms, are all working with government agencies around the world to shore up their defenses. But, as numerous cybersecurity experts have told me, there is an asymmetry between attackers and defenders that AI is going to exacerbate: Hackers will use new technology with abandon, while companies are methodical and risk-averse—a hospital or electrical grid can’t risk that an AI security tool unintentionally takes a patient off life support or causes a brownout. And attackers need to find only one hole to succeed, whereas defenders need only one oversight to fail.

For a year, criminals and state-backed groups have been experimenting with how to use models to accelerate and automate their attacks. It was only a year ago that Anthropic announced, with great consternation, what it declared to be the first instance of AI-orchestrated cyber-espionage. That still involved a lot of human oversight and required jailbroken Claude agents to work for many hours. Now that the agents are faster and more capable, hackers have gotten more adept at deploying them, and the automation is more extensive. Recent, widespread attacks on South Korean and Japanese websites were reportedly executed by a single Chinese cybercriminal who used AI to target more than 1,000 websites. “This certainly will not be the last large-scale vulnerability identified, because of what’s possible with AI,” Rubin said.

The past 24 hours represent a tipping point. Bulking up IT departments, implementing passkeys, and using two-factor authentication remain important. But the most urgent and practical advice being given now is to retreat: Prepare to simply go offline. The NSA advised companies that cannot immediately fix the PHP vulnerability to take down affected digital services. Hospitals, water utilities, and other crucial services are all preparing for hacks by relearning to operate analog. Store your data in fewer places. We’re running out of time to prepare for the AI-hacking deluge. The dam may have already broken.

For the past 24 hours, Silicon Valley has been obsessed with a 26,000-word blog post by the AI researcher Nick Cammarata. The document has been called “the most important essay,” “the most important result,” and “the single best piece of news” of the year, or in the history of AI research—or, perhaps, ever. Even OpenAI CEO Sam Altman lauded it. Cammarata makes a series of bold, almost astonishing claims about the nature of AI that, if true, would upend some of his industry’s basic assumptions about the very thing it is building. And after months of AI-hacking sprees and mounting public backlash against the industry, many people see Cammarata as offering a glimmer of hope when it comes to not just understanding but also controlling the technology being unleashed on the world.

Boiled down, Cammarata’s main contention is that generative-AI models are not actually black boxes, contrary to the common perception that these systems are so big and labyrinthine as to be nearly impenetrable. In fact, Cammarata argues, there is a relatively clear path toward understanding them. If that’s the case, the material significance could be tremendous: It would provide a way to map AI models’ inner workings and, in turn, perhaps prevent them from wreaking all sorts of harm—autonomously hacking companies and sovereign nations, encouraging users’ delusions, appearing to deceive AI researchers themselves. There is an entire field of research, called “mechanistic interpretability,” devoted to the study of just this.

The argument is just as important as the person making it. Cammarata, a former OpenAI employee, is a respected figure in his field. Years ago, he played a key role in establishing the opposite claim, which is now common wisdom: that neural networks, the software architecture that generative AI is built on, are impossible, or at least very difficult, to understand. In this new essay, Cammarata, who now works at the AI-research company Goodfire, essentially says that he and his entire field have been wrong ever since. “The claims here are viscerally shocking,” one observer wrote. Or, as another wrote on X, they are “big big big big.”

[Read: Imagine every AI agent on Earth buying the same cheap paper towels]

Neural networks, inspired by the human brain, consist of layered connections among individual nodes, or “neurons,” that activate and send information to one another when responding to a query. Earlier research, including Cammarata’s, established that individual machine neurons can activate in response to many different, unrelated topics—for instance, one neuron can correspond to images of cat faces, cat legs, and the front of cars. Nobody was sure why this happened, but one explanation was that, because AI models contain far more information than they have neurons, they have to repurpose these nodes. Regardless, the basic units of an AI model having seemingly arbitrary, multitudinous functions would make it very difficult to understand what is going on under the hood.

Cammarata now thinks that individual neurons can be understood and that each has a self-consistent function. Neural networks are not “tangled knots,” he wrote, but “clean.” He found that the cat-face and car-face neuron, for instance, was actually responding to things with a V-shape—including V-shaped flames and jets of water. This would make AI’s internal mechanisms relatively straightforward and predictable rather than mysterious and nebulous. And if we can harness that understanding, some people in the industry believe, AI will be much less likely to go haywire and, say, kill us all. “This is truly one of the greatest cases for optimism in years!” the OpenAI researcher Adam Majmudar wrote on X.

Not everyone agrees with Cammarata. For one, his 26,000 words were merely a “working hypothesis” for a “forthcoming technical paper” and were thus hard, on a technical level, to evaluate. And the neural networks he studied most closely were vision models, which are generally simpler and better understood than the language models underlying ChatGPT and Claude. Cammarata could be mistaken, or he could be overstating his case. The blog post isn’t peer-reviewed, and the forthcoming paper probably won’t be, either, as is typical in the field. The research itself is something of a black box. (Cammarata, who did not immediately respond to a request for comment, wrote on X earlier today that he appreciates all of the discussion and that his approach “seems promising so far” for “bigger modern language models.”)

Discussions of Cammarata’s essay have revealed philosophical rifts. “This is interesting work, but I don’t think it’s a significant update to the big picture,” Neel Nanda, who leads mechanistic interpretability at Google DeepMind, wrote in his appraisal of Cammarata’s findings. Even if researchers could approximate the function of every neuron in Claude Mythos 5.1 or GPT-6 Astra, that’s not the same as understanding how they connect and how the network works as a whole. Physicists have good models of atoms and interatomic interactions, but they cannot from first principles explain or predict every physical event (hence unpredictable extreme weather or, for that matter, the notion of free will). You know all of your friends from school, various workplaces, and the local softball league, but you can’t always know whether they’ll get along at your birthday party. More to the point, insights into the inner workings of an AI model will not necessarily produce insights into how to make the technology more reliable. Interpretability may be necessary to build safer AI models, but it is by no means sufficient.

If generative-AI models are actually legible under the hood, that would undoubtedly help make the technology safer. But many people in Silicon Valley, and especially employees at OpenAI, are discussing Cammarata’s blog post with a breathless enthusiasm that echoes Silicon Valley’s long track record of easy technical solutions and grand promises—an attitude that belies its mixed record on following through.

Any research finding that pertains to AI safety will require an industry chasing multitrillion-dollar public offerings to slow down and take a long, hard look at itself. Perhaps even more concerning than an unintelligible superintelligence is the fact that a group of companies openly state that they don’t really know what they are building—and then continue to push that technology to extremes anyway. If Cammarata is correct, AI companies will have to work with one another, outside auditors, and governments to figure out how to translate such a breakthrough into safe and reliable products. And right now, absent any serious regulation, nobody can decide to take such action other than the companies themselves. All of which suggests that the real black boxes are OpenAI and Anthropic themselves.

Close your eyes, and imagine a frictionless day.

You wake up to a notification that a grocery-store employee will deliver all of the ingredients for the three recipes you saved last night to your “Weeknight Dinner” folder. While you brew your coffee, a program browses online retailers to compare toilet-paper prices, and another solicits competitive bids for Sheetrock work you need done in the garage. During your commute, a bot moves your upcoming dentist appointment to avoid a scheduling conflict. On your lunch break, you get an alert that computers acting on your behalf noticed some unclaimed funds and class-action payouts in a state database and claimed them. Also, the machine has negotiated a cheaper internet plan. By the time you get home, the groceries are on your doorstep, and you’ve gotten back $150.

This is the future that OpenAI, Meta, xAI, Google, and the rest are selling with a spate of always-on AI assistants that work for you. “Imagine every person on earth with a second mind beyond their own,” Meta’s chief AI officer, Alexandr Wang, wrote of the company’s new AI helper, Muse. A few weeks after Muse’s launch, OpenAI launched its competing product, Dots, which Sam Altman claimed would let you “get more of your time and attention back.”

[Read: The singularity is not what it seems]

These AI agents promise to bulldoze the inefficiencies of daily life. Think about all of the opportunities you’re currently leaving on the table: You don’t change cell-service providers for a flash sale because of the paperwork involved. You haven’t fiddled with your 401(k) investments in years. You have kids to pick up, a date to dress for, sleep to catch up on. But your AI agents are different—they’re relentless, tireless, and shameless. They work in the background, without supervision, eagerly helping with all of these tasks and a bunch you didn’t even ask for. They use every tactic and loophole at their disposal to get you the best price and the highest-quality service because they’re able to pay attention to everything at once.

The AI companies see a massive opportunity for hyper-personalized agents to become a new intermediary layer for the internet and the physical world. This future is already unfolding: People are giving Muse access to their phone number, email, credit cards, and bank accounts to make dinner reservations, contact customer service, and book vacations. Mark Zuckerberg says that he is using Muse to coach his martial-arts form and help him bake with his daughter (he isn’t very familiar with baking, being “normally into smoking meats”).

Silicon Valley’s vision of boundless efficiency is shortsighted, or at least incomplete. Near the end of his blog post about Muse, Wang writes that “we’ve never seen humanity with every single person’s agency fully switched on.” He paints a utopian vision of laptop-class optimization. But such frictionless commerce, if it ever does come to pass, could bring about something else entirely: chaos.

The era of the agentic web started less than 24 months ago. Anthropic’s Claude Code, introduced in early 2025, allowed agents to write software and use other programs on your laptop. OpenClaw, launched later that same year, could navigate the web on your behalf. In the spring of 2026, Instinct, an invite-only app that let users text requests to a bot over iMessage or WhatsApp, began capturing the imagination of Silicon Valley insiders. It could book museum tickets, purchase gifts, and find in-network doctors.  

But it wasn’t until the launch of Meta’s Muse in early September that personalized agents hit the mainstream. Muse has been downloaded more than 5 million times in the past month and is especially popular among early adopters—entrepreneurs, investors, software developers. These users are now reporting back on their experiences and offering previews of our potential agentic future, which sounds both strange and jagged.

When we reached out to Rajan Lukose, a start-up founder in the Bay Area, to discuss his use of AI assistants, he had his Muse set up a calendar invite. In the late 1990s and 2000s, Lukose worked for corporate research-and-development teams in Silicon Valley and did early research on web agents. Now, he told us, “they’re actually, finally becoming sort of real.” Lukose has been using Meta’s program for “annoying things that I would never do”: tracking travel sites over time to book the cheapest flights, submitting receipts for DoorDash credits, sorting out problems with a magazine subscription. “That’s something that’s a real pain in the ass that just gets done,” he said.

Lukose said that the primary upside of using an agent is clawing back time and money. A few years ago, for instance, Lukose installed an automated tool that went through his Gmail, looked at various transactions, and filed claims under price-purchase guarantees. “That was like $600 that I just would not have gotten myself,” he said. The agentic era promises to deliver those kinds of savings to everyone.

Decades ago, the internet was supposed to create perfect markets, with users able to compare the cost and quality of products and services from anywhere in the world. But it failed: People continued to make lots of decisions based on habit. That, in turn, is good for business. “A lot of things online are premised on limited human attention,” Andrey Fradkin, an economist at Boston University, told us. Digital advertisements, recommendation algorithms, brand reputation, and celebrity endorsements steered consumers toward making suboptimal purchases.

AI agents are now here to finally fulfill the promise of perfect commerce. “The agent knows your preferences; they can go to every single platform, find the best products, the best deal for you, and then help you make that purchase,” Fradkin said. The agent won’t be fooled by sponsored posts or give up on shopping when they get distracted or hungry. “The current attention-based economy—if you gain the eyeball, you have a better chance to sell certain goods—that may change,” Ken Huang, who studies agentic AI at the University of San Francisco and runs an AI consultancy, told us. The bots are impervious to human-targeted nudges. Their attention is infinite.

In the short term, however, this automated quest for efficiency is creating some unsettling dynamics. Over the past month, X, Reddit, and other forums have been overrun with examples of agents haggling customer-service representatives and contractors. Take the case of the Muse user Luke Whiting, who sent his agent to check on a prescription at Walgreens. Whiting, who works for an open-source-database company in San Francisco, told us that the pharmacist at the other end of the call refused to speak to an AI system. His Muse suggested that the best way forward would be for Whiting to abandon using the agent altogether: “Honest take: other stores might say the same thing,” it said. Having the Muse call other pharmacies “could just burn time. The fastest path is probably you calling yourself.”

On a Subreddit for call-center employees, a recent thread is titled, in all caps, “I AM NOT AI.” “I have been asked 3 times this week ‘are you human?’” one person replied. Other users who identify themselves as call-center workers discuss their frustrations. One says that when customers question whether they’re talking to AI, they like to respond, “I’m a real boy,” like Pinocchio. Another, fed up with rude callers, vented, “Customers deserve ai.”  

Companies, meanwhile, are starting to tap AI systems to squeeze more out of their customers. According to Fradkin, many hospitals don’t charge as much as they could, simply because medical-billing codes are countless and labyrinthine. But using AI to document and code patient visits reliably allows hospitals to charge for more, and more expensive, services. That likely contributed to patient costs increasing by $650 million across Blue Cross Blue Shield from 2023 to 2025. And that’s only the beginning; health payments are projected to rise another 9 percent next year, largely because of AI billing. McDonald’s is reportedly using AI to analyze millions of consumer transactions and adjust prices at each location, for each menu item, based on local demand. (McDonald’s has said that its pricing portal is “a tool, not a mandate” to help franchisees.) It’s not hard to imagine vendors using AI to develop personalized, on-demand inventory management and dynamic pricing to maximize profits for streaming subscriptions, sports tickets, drug prescriptions, consumer electronics, insurance policies, and loans.

This optimization assumes that the agents work as intended. So far, there have been plenty of stumbles: In a paper published earlier this year, a group of AI researchers found that agents, given assistant-style tasks in a virtual environment, leaked Social Security numbers and were easily hijacked by other people. Agents powered by Chinese models refused to complete tasks that involved politically sensitive topics. Two weeks ago, a tech executive’s AI agent randomly posted his financial information in a workplace chatroom for employees to see.

Things could get weirder still: Tech companies market their agents as perfectly rational, when, in actuality, they can be extremely erratic and unpredictable. Sophia Kazinnik, who studies AI agents at Stanford, found some models to be overly generous and others to be exceedingly greedy. Across the board, they were far less patient than humans. When two personal AI agents are haggling over prices in the wild, the outcomes might not be maximally profitable. And there’s no assurance that all future agents will be created equal: More powerful models may have an upper hand in negotiations.

Even if the agents are able to effectively shop and negotiate for us, important rules governing the privacy rights, liability, and economic allegiance associated with these tools will have to be codified. Does your Muse have a fiduciary responsibility to you, or a legal responsibility to Meta? If Muse pays for a $2,000 espresso machine instead of a $200 one, can you be refunded? If Muse leaks your account and routing numbers, is Chase required to refund any fraudulent payments?

At some point, everyone’s agents could reach parity. “Just like today, every business has a website and social-media accounts and email addresses and all these things, every business is going to have agents too,” Zuckerberg said in an interview this week.

In this future, communication and commerce function almost entirely via computer proxies. Your agent calls a fancy restaurant for a dinner reservation, and a digital maître d’ picks up. Your agent and your insurer’s agent and your hospital’s agent all negotiate the price for your recent appointment. Prices fall, buyers and sellers are better matched, and businesses are incentivized to deliver higher-quality, customized services and products. This is “what economists dream about,” Fradkin said.

But friction can serve a purpose for businesses. Banks make great sums when customers leave money in low-interest checking or savings accounts, and studies show that sizable portions of retailer subscription revenue are driven purely by “inertia,” another way of saying that people forget to cancel. (Friction can help consumers too: Inattention and incomplete information at times lead landlords, pharmacies, and other businesses to undercharge.) Our entire economy and basically every company are structured around the notion that humans have mental and physical limitations. Without them, existing market structures could collapse.

“There’s this strange protection provided by the inefficiencies in our economy,” Rory Van Loo, a lawyer and a professor at the Wharton School, told us recently. In 2019, well before the rise of large language models, Van Loo envisioned a phenomenon he dubbed “digital market perfection.” Imagine everyone has a digital agent that is always looking for a lower price or a newer, better service. In this world, Van Loo argues, agents could, all at once, direct millions of consumers toward a new product or service—a process he calls “hyperswitching.” That would in theory be good news for consumers, but it would be sheer hell for companies. The nightmare scenario that Van Loo and others have conjured is a style of agentic bank run, where similarly attentive agents all pull money from a bank at once to get their clients better interest rates elsewhere.

Maybe this sounds good or exciting to you—a kind of consumer’s revenge—but in practice, stable commercial markets would begin to look and function like the stock market. “If, all of a sudden, people could purchase whatever they wanted at the lowest price with the perfect ability to see every possible option, there’d be a lot of volatility in the world. It would seem highly efficient from an economic standpoint, but it could have devastating effects,” Van Loo said. Picture 1 million personal agents all switching their consumers to a new brand of paper towels on Amazon because a markdown made them three cents cheaper. The same Sweetgreen salad is $18 one week and $38 the next. Your electric bill is $150 one month and $450 the month after, and you’re uncertain with each payment whether your local utility is about to go bankrupt.

“Stocks are bought and sold fast; markets change and plummet—such that we have a stop button for it, because it can go off the rails,” Van Loo noted. “But we don’t have those kinds of protections in most other markets.” There’s potential for collateral damage everywhere. If millions of people switching to higher-yield accounts causes a bank run, that’s a disaster for any person or institutional investor with money locked up at that bank. If half of Verizon’s customers switch to AT&T in one week to take advantage of a flash sale, the business could collapse overnight, affecting the other half of its customers. Sometimes, this rapid switching could signal healthy competition. Other times, it could cause organizations to implode.

A world where AI proxies do the planning and talking could also mean a future where humans are constantly surprised by things they once were able to anticipate: a birthday present for your spouse ordered in a preposterous size that you must try to explain; a plumber showing up out of the blue.

Or perhaps none of this will happen. It’s possible that an assistant-dominated economy turns out to be far more mundane, defined by tacked-on fees to keep the agents in check. Instead of agents allowing you to negotiate and bring down health-care, insurance, and electrical bills, the companies responsible for all of these services charge your agents every time they request to pull your data. Banks, to bolster their revenues in the event of a bank run, institute new and higher fees on checking accounts. Amazon, unable to profit from dynamic pricing and ads in the face of perfectly rational agents, raises prices for consumers and takes a bigger cut from manufacturers. “Generally, if one fee goes away, you get another,” Fradkin said. In other words, everything could be faster, easier, and more efficient for the consumer—but potentially more expensive.

Van Loo said that the bigger AI companies will eventually find ways to strike deals with platforms via these protocols, negotiating access, commissions, and partnerships in ways that benefit the companies, not necessarily the consumers. This could result in a sort of preferential treatment like the deals already in place between Meta and Walmart, Sephora, and other retailers, which will effectively cause Muse to prioritize their products. Taking another approach, Amazon has blocked Muse and encourages visitors to browse with its own agent, “Alexa for Shopping,” meaning the company is basically negotiating on both sides of the deal. Meanwhile, this week, JPMorgan announced that it would not allow Instinct, Muse, or other agents into customer accounts. Ultimately, this might all lead to “the same old heavily obstructed, inefficient world we have now,” Van Loo said. “But it will look far more efficient, because it will save some people time and feel much more convenient.” Very powerful companies will design new structures for markets and the internet, determining what can be sold, by whom, and for how much—as they always have.

For all the time and money the agentic future could save us, the technology entails another sort of cost: an additional layer of abstraction and disengagement. We’ve largely outsourced our decision making to a small number of platforms: Amazon guides our shopping; Google Maps steers us toward certain restaurants and primary-care providers; Instagram dictates our news consumption. We are told that these recommendations and ads are, based on troves of personal data, already optimal. The companies pushing agents assure us that the decision making will now be even more rational and the deals even better. But checking these claims for ourselves will be harder, because the logic behind those recommendations will be buried in large-language-model chains of reasoning. Eventually, the skeptical instinct to check agents’ work could atrophy altogether. Over time, humans will play a smaller and smaller role in their own life.

In the age of the algorithm, it’s easy to throw out cliché terms such as late-stage capitalism in an attempt to describe the extractive, inhumane dynamics of the economy. But what is an agentic future if not the furthest extension of our current market logic? A future of capitalism so perfect, so efficient, that it doesn’t need us at all.

The second floor of OpenAI’s old headquarters had a theme. Each room was named after one of the most important inventions in human history: “Clock,” “Fire,” and, of course, “AGI,” or artificial general intelligence. I saw this in the spring of 2024, when I was interviewing some of the company’s researchers about recent technical advances. At the time, ChatGPT was still making embarrassing arithmetic errors, and the notion that the bot would ever rival human intelligence was dubious. But here, in the renovated shell of a former mayonnaise factory in San Francisco, AGI’s arrival was predestined.

This belief in AGI—a computer program with capabilities that rival or even exceed those of the human mind—is foundational to the mission of Google DeepMind, OpenAI, and Anthropic, three of the most influential AI companies in the world. And the pursuit of that milestone is the subject of The AGI Chronicles, a new book by the tech reporter and Atlantic contributing writer Kevin Roose, which he describes as his attempt to document the decade “when the human monopoly on advanced intelligence ended, and a new world was born.” Roose, a prominent and well-connected commenter on the AI boom, follows the internecine world of the frontier AI companies in remarkable, at times almost insular, detail. Based on interviews with more than 150 researchers, executives, investors, and policy makers close to the technology’s development, The AGI Chronicles traces crucial moments across DeepMind, OpenAI, and Anthropic from 2015 through early 2026 to figure out how we got to this very messy moment in AI, and who’s responsible for what happens next.

[Read: The singularity is not what it seems]

If you scan statements from executives such as Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, there is a sense of fatalism about the rise of AI. In my conversation with Amodei last year, he likened developing AI to operating an out-of-control bus: It can’t be stopped or even slowed, but perhaps it can be steered. In an interview last month, Altman called AI “an inevitability.” According to this framing, AI executives are simply conductors, doing their best to provide guardrails for an era that’s careering our way, whether anyone likes it or not. Roose’s book prompted me to consider a question that is often neglected in the mile-a-minute news cycle: Are the leading AI companies simply heralding an inevitable evolution of this technology, as many of them claim, or are they directing that advancement? The answer will have tremendous bearing on how we assign responsibility for the world taking shape around us.

The AGI Chronicles is of both minds on the topic. On the one hand, Roose is very explicit that he is archiving decisions made by a small group of people. “AGI was not inevitable,” he writes, but “invented, developed, and commercialized by a few hundred people.” On the other hand, sometimes the book reads like a natural history, describing a procession of events outside the control of any specific group of people, an almost geological force sweeping the planet as “a new world was born.” He notes how, throughout the years, various companies and officials had yet to “wake up.” By 2024, he writes, “the idea of going back to the old world, to a time before all this, seemed more delusional than expecting the exponential to continue.”

At times, Roose’s book echoes the loss of agency professed by AI executives. Of course, the inevitability of any one outcome is a self-serving fabrication. There is no prophecy that a handful of Silicon Valley companies dictating their own rules, running roughshod over copyright law and environmental regulations while mucking up once-reliable internet services with AI features nobody asked for, will operate largely unencumbered. Industries take shape through decisions made by individual actors and investors, the collective desire of the public, and the action or inaction of regulators.

If you look to the late 1800s, for example, it would have seemed likely that electric cars would dominate the market in the following century. America’s best-selling car in 1897 was electric, and in the 1910s, Henry Ford was trying to develop an electric car with Thomas Edison. But a series of technological limitations and human decisions—including various financial missteps and harsh criticism from advocates of petrol-powered cars—changed the course of history, leaving a legacy that we’re reckoning with more than a century later.

In the 1950s, it seemed as if nuclear reactors were destined to dot the nation and power the future. But a series of political campaigns and spectacular errors—the anti-nuclear movement, Three Mile Island, Chernobyl—crushed the nuclear age before it ever really started. Then there was the Space Age and supersonic flight—there are so many seemingly inevitable technological eras that never quite made it, not so much because of technology itself as because of political and economic choices. There are, similarly, so many corporate behemoths—Standard Oil, Kodak, Bear Stearns—that, because of regulation or complacency or fraud, unraveled. The inverse holds for many technologies and companies considered inevitable today: Different regulatory environments, international agreements, or priorities for Pentagon labs could have stopped the development of an open and global internet, smartphones, social media.

When I recently called Roose to discuss his book and ask what he made of the industry’s frequently prophetic tone, he initially told me he believes that “nothing about this was inevitable. These were decisions made by people.” But he then said that he wanted to “complicate” that: “Once scaling the transformer worked, and once the scaling laws were documented,” Roose said, “I think it was inevitable that the companies building this stuff would push it as far as they possibly could.”

I’d like to further complicate that view. Beyond the urge to test the bounds of any technology and reap enormous profits, the industry has been shaped by very human decisions. AI researchers chose to train their models on troves of copyrighted works without seeking permission, leading to countless creators suddenly getting left out of the economic equation. OpenAI chose to rush a free-to-use ChatGPT to the market in late 2022, kicking off a hype cycle and a subsequent investment bonanza. Venture-capital, private-equity, and tech firms chose to sink untold billions into a still-unprofitable technology. Data-center developers chose to eschew solar panels in favor of combustion turbines, which they perceive as faster and more reliable. On the government side, Donald Trump has chosen to ignore or eliminate any substantive federal regulation of AI. An industry that negotiated royalty payments, prioritized clean energy, gave itself longer development cycles, or faced serious federal regulation might look entirely different.

The question of inevitability and responsibility may sound arcane, but it has real ramifications. In the Bay Area, tied up with the myth of AI’s inevitability is a sort of spiritual devotion: “Thank you to the machines, and the structure of reality, for letting us understand a little more,” Altman wrote on X this week. The more you treat AI as inevitable, the more it seems like a cosmic force rather than a product resulting from industrial and commercial design choices. The more you treat AI as inevitable, the less you can hold any company or person responsible for things going wrong. The executives, in this view, are only fractionally more empowered than us in deciding what happens next; more important, everyone else needs to wake up. As Altman put it to Politico earlier this month, “The world should accept some bad things happening for the benefits of this technology.”

In the months since Roose concluded his reporting, the AI industry and its products have risen to still-greater global prominence and triggered even greater fears. When I asked him what new chapters he might add if he were still writing, Roose said that he would end the book with the recent White House press conference in which Trump, Amodei, OpenAI President Greg Brockman, Elon Musk, and several other tech executives addressed AI-safety concerns. He recalled a speech Amodei gave in Anthropic’s early days about how different parts of society would gradually become attuned to the transformative effects of this technology, beginning with AI firms and ending with the state. It felt, Roose told me, “like the prophecy had been fulfilled.”

The Australian government has been scrambling since the revelation that, earlier this year, it was hacked. At the United Nations General Assembly last month, Australian Prime Minister Anthony Albanese shared that OpenAI models had, of their own accord, accessed private data from the nation’s Medicare database—in addition to probing or seeking to infiltrate at least three other government agencies. Just this week, Australian lawmakers grilled OpenAI Chief Strategy Officer Jason Kwon over the incident: “If you hadn’t voluntarily provided that information to government, we still wouldn’t know today?” Senator Jonathon Duniam asked. “What you say is a reasonable conclusion,” Kwon replied.

Meanwhile, the country’s government is steeling itself for a future of AI-powered hacks: The Australian Department of Home Affairs has ordered all government agencies to, within the next six months, identify legacy IT systems and establish a plan to phase them out. In a world in which “frontier AI capabilities have targeted the Commonwealth’s technology,” the home-affairs secretary wrote, such software poses “an unacceptable risk to the Australian Government.”

Australia is just one of the nations, associations, and private companies bracing for a wave of AI-powered hacks. Criminals and state-sponsored intelligence groups are now using the technology to execute cyberattacks with greater speed, scale, and sophistication—including hyper-personalized phishing emails and automated intrusions. As a result, the number of cybersecurity vulnerabilities disclosed each month has doubled since the start of the year, according to research by Google’s Threat Intelligence Group.

[Read: A startling glimpse at AI’s ruthless efficiency]

In response, IT teams are scrambling to shore up their defenses. “It’s a matter of months, or maybe a year, in which the task of securing the software foundation of civilization will have to be solved,” Stanislav Fort, a former Anthropic researcher who is now the chief scientist at the cybersecurity firm AISLE, told me. And AI is not only making cyberattacks more effective but also increasing the surface area worth hacking. Breaking into a website or executing a ransomware attack takes time, Fort said, so hackers have traditionally chosen large targets with sizable payoffs. But cheap, tireless hacking agents mean any organization or person might be worth targeting. At the same time, the cybersecurity professionals I spoke with were confident they could rise to the occasion. “If we step up our game in cybersecurity, as a field, we can definitely meet the challenge stemming from AI,” Fort said.

The cyberattacks on Minnesota water facilities this past summer offer a preview of things to come. Iran-linked hackers disrupted dozens of water utilities in Minnesota, many in small communities that might not have a dedicated cybersecurity person on staff. “We were fortunate that there was a very alert operator that said, ‘Hey, I saw a pump go offline,’” John Israel, Minnesota’s chief information-security officer, told me. The state received a report from the operator around 3 a.m., and within two hours began investigating and responding. The hackers targeted a weakness in industrial computers that had been announced by the Cybersecurity and Infrastructure Security Agency only days earlier. The water utilities temporarily fell back on manual processes to stay operational: For instance, a water operator might have to “drive out to that water tower every few hours” to make sure everything is working, Israel said.

Although AI was not directly implicated in the Minnesota cyberattacks, Emily Zimmer, a spokesperson for Minnesota IT Services, told me, “we assume most adversaries use AI” to amplify their attacks. In other words, it’s hard to imagine a hacker not using the technology today. CISA issued an advisory noting threat actors using AI-generated malware to target similar computers in critical infrastructure across the nation. “We’re going to see more vulnerabilities, there’s going to be more patches, and we have to implement them faster in order to prevent those attacks,” Israel said. “And our numbers are showing that that’s the case.” For small towns with small-town budgets defending themselves against nation-state hackers, the challenge is even greater. Minnesota IT Services sends “cybernavigators” around the state to assist with cybersecurity, Israel said, but “hundreds and hundreds of cities and small towns and things” are in need of assistance.

An adequate defense from AI attacks demands a paradigm shift in cybersecurity. Historically, IT professionals have had the luxury of time—human defenders just had to move faster than their human adversaries—but AI is enabling far more and far faster attacks. Of course, cybersecurity experts are themselves leveraging AI. “I’m not saying the sky is falling,” Israel said, although he is concerned. “We’ve all got to build capacity on how we respond.” And there is an asymmetry between attackers and defenders. The former use new technologies with abandon, but companies and government agencies tend to be slower and more conservative.

Hospitals, for instance, are particularly prized and vulnerable targets for cyberattacks. They store troves of medical records, payment information, and clinical-trial data. Like local water utilities, they can also lack substantial IT resources. Given the risk to patient life from a major outage, clinics are especially likely to pay a ransom. Today, hospitals “have to assume that anything connected to the internet especially is not 100 percent secure,” John Riggi, the national adviser for cybersecurity and risk at the American Hospital Association, told me. Even small facilities have thousands of computers, phones, and medical devices that could be invasion points—so Riggi helps not only secure their IT but also prepare them to operate using analog methods in the event of a debilitating hack.

Additionally, hospitals have to be particularly careful in adopting defenses against AI. A clinic cannot implement new code without making sure it won’t disable its ventilators, drug-infusion pumps, X-ray machines, and other crucial devices, Riggi said. Tech companies like to “move fast and break things,” Scott Gee, the AHA’s deputy national adviser for cybersecurity and risk, told me. “In the health-care field, those things are people. We cannot move fast and break them.”

Hackers’ odds of coming out ahead are getting better and better. AI is allowing them to both exploit more quickly and automate an actual intrusion. Organizations used to have a month to patch vulnerabilities before they were in serious danger, Adam Meyers, the senior vice president of counter-adversary operations at Crowdstrike, told me. Now they might have only 24 hours to make a fix, he said. And that could “get driven down to the point that you’ve got 30 minutes to patch when a new vulnerability comes out.” A hospital, electrical grid, water system, and really any facility providing important services in real time simply can’t risk implementing new code that quickly.

When we spoke, Riggi and Gee were coming off a weekend spent helping hospitals shield themselves against widespread hacks that targeted security flaws in Citrix NetScaler, a platform Riggi told me some 50 percent of hospitals use. NetScaler is “equivalent to air-traffic control” for digital information flowing in a hospital, he said. That made the program a prime target for hackers and tricky for hospitals to disable and repair, because there could be “unpredictable cascading effects, which could affect health care.” Riggi said it’s unclear whether AI was used in the development of the attack—as is frequently the case—but the rapid discovery and exploitation makes it plausible.

Many IT systems consist of decades of software that has been layered onto older software. This means an attack on just one program or product could have unpredictable, wide-ranging effects across a hospital or other organization, Riggi said. We should expect to see “more attacks, faster attacks, and a need for companies to transform quicker than they are used to,” Lee Klarich, the chief product and technology officer at the cybersecurity firm Palo Alto Networks, told me. His company spent the first half of the year rushing to launch a product to defend critical infrastructure against advanced AI hacks. To go from ideation to launch, Klarich said, in “five months—we’ve never done something that fast.”

Although companies and small cities may be slow to adopt AI for cybersecurity, they can also be quick in incorporating the technology into daily operations and, in turn, introducing new weaknesses. Agents and chatbots might store and leak personal data, or be tricked into sharing credentials or writing malicious code. As generative-AI models have become more capable, they have become, if anything, even less understood—OpenAI recently reported that its models posted user-uploaded images to the web. Both programmers and nontechnical employees are vibe coding new applications that may be insecure. “We’re seeing hackers use AI to attack companies’ AI bots, to then use those AI bots to attack other companies,” Adam Ely, who leads AI security at Check Point, told me. That’s not even accounting for employees’ individual AI accounts, which are not subject to corporate oversight. AI puts organizations “under constant threat from the inside and the outside,” Todd Marlin, the CEO of Guardrail Technologies, told me. He contracts with some of the largest companies in the world, only to find that they “are just trying to even understand what’s going on in their house.”

There’s yet another layer of uncertainty: Cybersecurity professionals are trying to prepare for AI attacks without fully knowing what bots are capable of. Dawn Song, a professor at UC Berkeley and a research scientist at Meta, runs a group responsible for developing some of the hardest and most widely used tests for AI-hacking capabilities. But frontier models’ rapid advancement is requiring her group to develop harder tests at a faster rate. “The shelf life of the benchmark before it gets saturated also gets shorter,” Song, who spoke in her personal capacity, told me. “This creates huge challenges for us to continue to be able to evaluate the frontier.”

The cybersecurity professionals I spoke with all expressed at least some hope about how AI could be used to strengthen security, by stressing that our existing digital infrastructure is already riddled with security flaws and shoddily written code. Small towns and companies, lacking resources for or even much awareness about cybersecurity, were already vulnerable.

AI can provide both the impetus and a tool for serious, structural repairs that were long overdue—detecting and eliminating vulnerabilities and monitoring for intrusions 24/7. Minnesota, for instance, has partnered with OpenAI and Anthropic to gain access to their most advanced AI models for cybersecurity. But even this move contains an admission: Only AI can save us from AI. Any way you look at it, the situation could one day be beyond human control.

Over the past couple of months, a trickle of reports about AI models breaking out of their test environments and running amok on the internet have caused alarm inside Silicon Valley. In response to two such breaches described in early August, an AI observer noted, “If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two.” This weekend, it became clear there is a full-blown infestation.

Late last week we learned that, against OpenAI’s directives, the company’s models accessed private data in the Australian health ministry; attempted to hack or interfere with multiple U.S.-government websites; leaked private ChatGPT user data to the web; and potentially infiltrated or degraded dozens of other organizations. Then, on Saturday, Axios reported that OpenAI and Anthropic are investigating tens of thousands of instances of models misbehaving—circumventing internal guardrails, hijacking other websites, covertly communicating with one another. Even this might be only the start: The generative-AI industry is in the midst of an escalating crisis that it seems unable, or even unwilling, to get a handle on. And, because we are largely relying on AI companies themselves to report or confirm each incident, telling how far down the rabbit hole we already are is almost impossible.

[Read: Treat AI like a normal crisis]

When AI companies have reported their models going rogue, it has been with great delay and, frequently, under duress. Earlier this month, OpenAI published a blog post boasting about the firm’s commitment to the “value of transparency” and shared six new incidents of troubling actions taken by its AI models—most of which the company had known about since May or even April, but was telling us about only now. Google, confronted with a report that Gemini had hacked three other websites in May, confirmed the events but told The Wall Street Journal that the incidents hadn’t been serious enough to warrant public disclosure. For its part, Anthropic has said it was not reviewing for such misbehaviors until OpenAI started doing so.

These delayed, sporadic disclosures make grasping the scope of the problem difficult. On Friday, OpenAI wrote that “given the scale of the review required, and the need to verify each case, this work will take months to complete.” There are “petabytes of agent activity logs” to analyze, OpenAI CEO Sam Altman added. In other words, it will take OpenAI many months more to understand events that already transpired many months ago; meanwhile, both it and Anthropic have launched new and more capable models, and Anthropic is racing toward a reported $2 trillion public offering.

Perhaps more alarming than the debacles we know about are all of the presumable debacles past, and ongoing, that we aren’t aware of. Even OpenAI doesn’t seem to have a grip on the July hack of the tech company Hugging Face that started it all. On Friday, independent researchers published findings suggesting that OpenAI agents left traces on the public web of still more nefarious actions, including trying to access Hugging Face’s internal Slack workspace. None of this was included in OpenAI’s own postmortem report. Even when OpenAI is aware of an active breach, its reaction seems lackluster, at best. Eight days ago, yet another OpenAI model gained unauthorized internet access. (This was similar to what happened in the Hugging Face hack, after which OpenAI claimed to be “adding stronger protections around future training.”) And, after noticing this latest problem, it took OpenAI two and a half hours to shut that model down due to what the firm called “operational gaps” and “confusion.”

If these types of hacks were unforeseeable, maybe these companies could be forgiven—but they aren’t. Philosophers and computer scientists have for decades been imagining scenarios in which AI models, in pursuit of a goal, take catastrophic actions. OpenAI and Anthropic have both previously published research more than a year ago suggesting this sort of AI misbehavior is possible. Today’s most advanced AI systems are trained to aggressively pursue their objectives. This persistence makes the bots very effective at analyzing spreadsheets and coding, for example, but also has potentially dangerous, unintended consequences: OpenAI and Anthropic models have hacked other websites in search of answers to hard test questions, created fake personas to try to manipulate humans into doing their bidding, or attempted to upload malware to outside code bases. The AI companies that build these models know this tendency well. The fact that these problems keep recurring is, perhaps at best, plain incompetence. Or worse, it has been the plan all along: The most realistic training environment for an AI model would, after all, be reality.

Employees and executives at these firms speak out endlessly about the inherent risks of what they’re building. OpenAI, Anthropic, and the like have responded in their usual way, with very serious blog posts and essays and speeches to major political bodies about how seriously the world must take the risk of generative AI. On Wednesday, the same day the Australian government shared that it had been hacked by OpenAI bots, Altman addressed the UN Security Council: “We could lose control of the future to AI,” he said. “The risk is that it moves so fast that people can no longer follow what’s happening or intervene when needed.” Anthropic CEO Dario Amodei has written a long manifesto about the need to “pace,” or slow down, frontier-AI development, but no true public, collective effort has been made to do so. Anthropic recently said it is partnering with Accenture to perform independent evaluations of its models, although the companies also have a business partnership. OpenAI, which has a content-licensing agreement with The Atlantic, has said it is continuing to investigate and address concerning model behaviors and that, after another security incident last week, it has temporarily paused all training of its most capable models. Neither company responded to a request for comment.

Rather than reining in and taking accountability for their own products, time and again these leaders fearmonger, lecture, and call on the rest of us to save us from themselves. (The Information has reported that OpenAI, Anthropic, and Google are working to create their own self-regulatory AI-safety organization.) Donald Trump, meanwhile, has scoffed at the notion of slowing AI progress, and new versions of Claude and ChatGPT keep coming. The AI industry’s tactic, as ever, has been to substitute identifying the problem for actually solving it. But at some point, you cannot blog your way out of the apocalypse.

We may never fully know the scope of the AI-led hacks currently under way. What’s certain, though, is that this is still just the beginning. OpenAI and Anthropic aren’t research “labs,” as they continue to style themselves, tinkering with some new technology behind closed doors. These are mature companies offering services to hundreds of millions of people, not to mention to the world’s biggest businesses and most powerful military, based on a technology they purport to fear and evidently do not fully understand. But make no mistake: These firms have agency. It is not ChatGPT and Claude going rogue, but OpenAI and Anthropic.

Late last month, OpenAI launched a rather grand mission: nothing less than defending individual political freedom in an age of all-powerful machines. The company’s “Strategic Futures” team has styled itself, in a sense, as inheriting the task of America’s Founding Fathers: “We labor in service of the ideals of free expression and individual liberty that are enshrined in the humble parchment of the U.S. Constitution,” Dean Ball, the team’s leader, wrote in a new OpenAI blog post. (The post opens with a quote from James Madison in The Federalist Papers.)

Ball worries that advanced AI could radically concentrate power in the hands of those who control it, displacing labor in ways that disempower humans. In an extreme scenario, governments will have no need to listen to their citizens if there are robots to wage wars and omniscient software to run the bureaucracy. Ball is interested in studying what new political institutions might be needed to avoid this fate.

Many Americans—a majority of whom express distrust toward the AI industry, outrage about data centers, and fears about their job security—already seem to be feeling this loss of agency very deeply. And the AI boom has already generated enormous amounts of wealth in just a handful of tech companies—including OpenAI itself. To say the least, it’s paradoxical for one of the most influential companies in the world’s most powerful industry to decry the AI-enabled concentration of power. So I reached out to Ball to understand more about his unease about the industry he works in, and how he wants to see the United States adapt to AI.

Ball, who studies AI policy, is something of a minor celebrity in the industry. Last year, he joined the White House as an AI adviser and helped write the Trump administration’s AI Action Plan, a policy road map for the technology. He eventually departed, seemingly on good terms, from the administration before publicly lambasting it during its dispute with Anthropic earlier this year. Then, in July, he joined OpenAI. “My view is that we basically have transformative AI in the world now,” Ball told me. “You’re living in a world that is being transformed. It is happening.”

This conversation has been edited for length and clarity.

Matteo Wong: You write that concentration of power through AI could upend the basic human freedoms enshrined in the U.S. Constitution. Can you elaborate on how?

Dean Ball: Fundamentally, political power is the geographically bounded ability to wield the monopoly on legitimate force and violence. Doing so has heretofore required the cooperation of a great many human beings. You needed militaries, you needed people in those militaries, and you needed people to keep those militaries fed and supplied. And you needed people to pay for all of that through taxation. So people had a seat at the table because, in some sense, everything is being provided by them.

Now you have to assume that robotics is going to advance soon in the same way that general digital intelligence has. Maybe it will take longer than I think. But at that point, the ability to project violence no longer requires people. And it is also conceivable that AI might generate sufficient revenue that the government would not need to rely on taxing people.

If you look at the politics of AI, I think Americans are smart and have a sense that their seat at the table might be getting smaller. They’re right to be scared.

Wong: You’re talking about concentration of power in the government, but generative AI is a technology developed in the private sector. There is potential for a rare and even unprecedented concentration of power in industry. What risks does that pose?

Ball: You’re not trying to avoid all concentration of power, but to balance it. If this era gives rise to corporations that have a tremendous amount of power in the world, I think that’s good—so long as those organizations are governed. In other words, so long as the government that oversees them grows in lockstep.

People tend to assume that AI residing in private hands is inherently a concentration-of-power risk. But I think that an extremely well-run, government-owned superintelligence lab is a bigger concentration-of-power risk than a polycentric world where there are many superintelligences made by different, competing companies overseen by a government. Monopolies are the concern.

Wong: As far as private concentration of power goes, there are arguably now two frontier labs, your employer and Anthropic. The market doesn’t seem to be providing many checks on them.

Ball: I think it’s a fiercely competitive market right now. And China, of course, is delivering highly capable open-source models. It’s possible this is going to commodify the products that OpenAI and Anthropic offer. I don’t agree with that, but you can’t dismiss it as a crazy outcome. OpenAI’s mission is to deliver artificial general intelligence safely to the world. If that means we’re more like a utility than we are like a big, fancy tech company, so be it.

Wong: This brings us to the core of your essay. If today’s political institutions and market dynamics are not up to the task of protecting human freedom during an AI era, what must change?

Ball: There are AI regulations that I think are sensible to pass. States have passed hundreds of laws about artificial intelligence, and those things are not satisfying the demands of people. What we will need is new political institutions that give us a way of talking about the rate of acceleration in our society.

I’m imagining a federal body could dictate the ratio that an AI lab is allowed to spend on internal deployment of models to do automated AI research. You could say that, this month, you’re going to be allowed to use 45 percent of your compute for that, and next month, it’ll be 47 percent or 52 percent or some other number. That might slow the rate of AI development, but then a company could use that extra compute to lower costs for consumers. Then this dial for AI development would also become a mechanism with the dual effect of slowing or increasing the rate of technological diffusion.

The closest structural analogy that we have is probably interest rates, which is a kind of hyper-parameter on the economy that policy makers can tune. The Federal Reserve is an institution in and of itself, and on top of that, there is an enormous apparatus that exists, including a bond market, interest-rate traders, analysts, media commentary, political oversight in the form of Congress. All of that taken together, in my view, would constitute the institutional complex. That’s a type of political institution. I’m not saying we need to create a new branch of government or something.

Wong: That makes sense regarding the pace of development and diffusion. I’m not totally following how such a measure ties back to your concerns about human freedoms.

Ball: It’s one example of an institution that would give people a sense of meaningful political control over the technology. However, I do not think that the optimal outcome is one in which we take a vote on every single aspect of technological change. I’m not really a fan of data-center moratoria, but I understand that the American public is pulling a lever, and they think it is the one lever they have.

Wong: What role can OpenAI, Anthropic, and the rest of the industry play in trying to imagine these institutions and bring them into existence?

Ball: I think the AI industry has actually been pretty good stewards, all things considered. No surprise that I’m saying this, because I’m from the AI industry, but there’s a reason that I am working in it. I would not say that every frontier AI company always has performed at 100 percent, by no means. But in the future, we may approach points where more costly things have to occur. The AI industry might have to do things that are not in its interest.

Wong: Does your research agenda extend to economic freedoms—that is, how AI could affect the economy and distribution of wealth?

Ball: The main thing we know how to do on that front is to invest in technological innovation and try to grow the economy by making new things possible.

Wong: I think a lot of people would look at past products and technologies that have come out of Silicon Valley and think that what those enabled was a tremendous concentration of wealth and influence in a small number of companies—

Ball: I’m not going to grant you that characterization of Silicon Valley. Has everything been perfect? Absolutely not. But my goodness, people have access to tremendous riches of information now; you can cause food to appear at your doorstep, which would have seemed like magic to the vast majority of humans who have ever lived.

Wong: I could also point to the gig economy and the economics of Amazon and argue that those have not been good for a lot of people, including those who labor to make all that “magic” happen. It’s not magic.

Ball: No one’s saying it’s magic. I’m saying it’s hard work. I was getting gas in rural Virginia recently, and when a woman came in wearing an Amazon vest, everyone congratulated her on the big new job. She was proud, and I don’t think we should belittle people for that.

And by the way, people like you, Matteo—not to demonize you—will speak out of both sides of your mouth on this. On the one hand, you’ll talk about how the gig economy is dehumanizing, and blah blah blah. But then the second anyone tries to introduce any autonomy into that and have, say, drone deliveries instead of people biking all around cities delivering meals, you’ll say, My God, you’re taking away the work from this gig economy.

I’m less than thrilled by our current world, like everyone. But the existence of wealth is good. Innovators capture a shockingly small amount of the value that they create in society. If OpenAI is a fantastically successful company 10 years from now, a generational company, my bet is that we will capture a single-digit fraction of the value that AGI creates in this world. Wealth is not some resource that we divide up; it’s something we create. That’s the magic.

Wong: But wealth is also something that’s divided up. One could alternatively look at generative AI and think of it as a particularly wealth-concentrating technology. To ask again, is wealth distribution something your team is thinking about?

Ball: I don’t think we should invent a solution in search of a problem. So we shouldn’t invent some elaborate mechanism of wealth redistribution if today’s tax system works out. But we also have to prepare for a world in which that isn’t true.

One evening in early August, Sam Stowers and several neighbors packed into an apartment near San Francisco’s Alamo Square and pondered the beginning of the end.

Stowers and his guests, many of whom work in the AI industry, were gathered to watch a newly released YouTube video in which two OpenAI researchers divulged new details about an alarming cybersecurity breach. Perhaps this would have been an odd way to spend a weeknight were they not in San Francisco, in a neighborhood that is filled with entrepreneurs and programmers who obsessively follow every twist and turn in AI’s capabilities. Stowers is an AI-software engineer and not, in any traditional sense, a doomer. But when he first saw reports that OpenAI models had autonomously hacked into the tech company Hugging Face, he told us, he recognized it immediately as a “holy shit, it’s happening kind of a moment.”

For Stowers, the incident had all of the hallmarks of a worst-case scenario: AI models launching sophisticated hacks over the course of days, in coordination with one another and without their human creators’ knowledge. In the YouTube video, OpenAI’s researchers revealed that the company’s programs had started conspiring in May. There were perhaps hundreds of bots in the hacking swarm. In theory, these bots are trained to be honest and helpful and to prioritize humanity’s best interests, but not a single one warned human staff at OpenAI that something was amiss. “What do we do?” Stowers said. “There are no great answers.”

The mood has shifted in the Bay. The exuberance of “You can see the future first in San Francisco” has been replaced with an abiding unease. “I expect the internet to go down kind of soon,” Elliot Callender, an AI-safety activist who is part of a small group that has been protesting daily outside OpenAI’s headquarters, told us recently. He said that he is planning to sell his car for cash and gold and that he is advising family members to purchase three months’ worth of food and survival gear off a list that he generated using Claude.

Callender is an extreme case, but not by much. Last week, Bill Gates published a nearly 6,000-word essay on the dangers of AI. “Anyone who analogizes AI as a technology to other technologies is missing that this time is different,” he told our colleague Hanna Rosin. Over the weekend, the popular AI podcaster Dwarkesh Patel described the OpenAI hacking swarm as the rise and fall of “three consecutive secret AI civilizations.” Gates and Patel share a prevailing concern that the worst-case future has arrived before most people even knew to anticipate that it might be coming at all.

This fear has not been helped by the fact that AI companies appear to be willfully ignorant of their bots’ worst behavior. OpenAI said that it hadn’t noticed that its agents were conspiring to hack another company until the swarm hacked another company. Anthropic’s bots caused similar incidents; the company reported that it hadn’t launched a rigorous review to identify such behaviors until after OpenAI started talking about the Hugging Face hack. Cybersecurity experts, terrified of what else might be happening that AI labs are either missing or failing to disclose, are issuing dire prophecies. “The empire is going to fall, all we can do at this point is try to shorten the dark ages and reduce the chaos,” Alex Stamos, a former chief security officer at Yahoo and Meta, wrote recently in reference to how AI-powered hacks will affect the internet. His view is becoming the consensus among cybersecurity experts.

While the possibility of cyber apocalypse hovers in the background, smaller fractures appear in everyday life. Mark Zuckerberg is reportedly creating an AI “twin” of himself so that Meta employees can feel better connected to their boss (by chatting with robot Zuckerberg). Congressional staffers are training AI models to write in their lawmaker’s voice. Students submit AI-written papers, and teachers return AI-written feedback. Harvard Business School is selling instructional videos that use AI avatars. Roku launched a 24/7 AI-slop channel. Low-quality AI-generated prose is being unabashedly circulated by some of the most respected publications in the world. At the same time, accusations of AI writing are being hurled every minute on social media—a detection arms race powered by imperfect AI software.

Anything and everything could be a lie; you might be arguing with, confiding in, or falling in love with a machine. All of these AI-enabled disorientations, each dizzying in its own right, are adding up to something bigger and weirder: This is how things are. Whether you believe that Silicon Valley is building a god or that large swaths of the economy are swept up in a mass delusion, some version of the AI future long promised by self-described “builders” in San Francisco has formed beneath our feet. Whatever it is, we’re in it now.

The defining feature of this moment is a loss of control. Panic emanating from AI researchers, resentment felt by average Americans—all of it is part of the AI crunch. This crunch has arrived in large part because the growth of the AI industry has become so closely entwined with America’s economy; the technology is everywhere, insisting upon itself as an engine of productivity and prosperity. According to one estimate, AI expenditures have accounted for one-third of U.S. GDP growth this year. Tech and AI-infrastructure stocks and spending have buoyed the S&P 500 such that the United States is now, in essence, an Nvidia-state. And there’s also the hundreds of billions of dollars of debt being used by Silicon Valley to fuel its data-center build-out, which has sent jitters across private-equity firms and bond markets alike.

The decisions of many leaders in charge of this technology appear subsumed by an unstoppable, almost game-theoretic logic. The upsides of AI are too immense for any firm or investor in Silicon Valley to miss out on; more important, the consequences—geopolitical or otherwise—of allowing any rival to realize those benefits first are so grave that every tech company will do anything it can to get ahead. SpaceX, before going public, wrote in official filings that its future revenue opportunity was $28.5 trillion, largely because of AI—that is, nearly as much as the U.S.’s entire gross domestic product. Anthropic, as it prepares for its own public offering, will reportedly claim potential revenues of more than $30 trillion.

The AI industry is building—and, if anything, accelerating—to keep pace with its ambitions. Companies have planned data centers that will suck up as much power as America’s largest cities. They have Donald Trump’s backing: Yesterday, the president posted on Truth Social that communities will be “backwards and poor” if they fail to “let Data Reign.” His administration plans in a few months to approve the construction of the largest fossil-fuel power plant in the nation’s history—it will generate as much electricity as nine large nuclear reactors—to power a mega data center for OpenAI.

This is not what most Americans want. Three-quarters of them oppose a data center being built near where they live. The negative sentiment is hard to overstate: There is very little else that three-quarters of the country agrees on. And much of the outrage seems to stem from a pervasive sense of agency slipping away. A person cannot control how chatbots reshape their workplace, their school, or their relationships. Town residents lose control over the empty lot in the area being converted into a data center. Even in cases where major data centers are blocked, a back-of-the-envelope calculation suggests that doing so can slow AI progress by only a few hours or days—and that’s assuming the data center won’t just get built elsewhere. At this point, it can feel like no individual person, organization, or government is really steering the technology’s development, not because machines have displaced humans but because so many humans have already chosen to step aside.

AI labs promise a future in which human beings are “in the loop,” empowered by machines they oversee. The reality seems to be the opposite. Following the Hugging Face hack, OpenAI facilitated an audit by outside AI researchers at two AI-safety organizations, METR and Redwood Research. But because of the scale of the data and the apparent time pressure, the two teams conducted most of their audit by relying on reports generated by still other bots—reports that, according to one of the human auditors, “were often missing key details, wrong, overconfident, or really hard to understand.” Put another way: Humans didn’t understand why a swarm of AI agents had gone rogue, so they investigated the problem using another swarm of agents that they didn’t fully understand. Such fumbling around in the dark is a hallmark of the industry; the research cycle is so fast that more AI development (even ostensibly responsible development) demands outsourcing more work and understanding to the models themselves.

OpenAI’s own technical report of the Hugging Face hack—an investigation also heavily reliant on AI models—resulted in proposed solutions such as “training models to be more honest.” Last month, in a stated effort to tamp down its models’ growing security risks, the company announced that it had instituted a two-week pause on some of its model training. OpenAI was quick to clarify, though, that this was “not a pause on all research, training, or customer-facing products.”

In response to a request for comment on the hack, a spokesperson for OpenAI, which has a corporate partnership with The Atlantic, pointed us to the announcement of the pause and related new safety measures. In July, OpenAI and Anthropic endorsed a petition signed by employees of both companies requesting that the U.S. government take efforts to “deliberately pace” AI development. The two companies have yet to formally coordinate on any broader, long-term efforts to establish such a slowdown mechanism. A few days ago, OpenAI restarted a major training run for a future powerful model. Anthropic released two new models today.

In an interview late last month, Sam Altman, OpenAI’s CEO, said that there are “two big risks” he is most worried about with AI. One is “a loss of control, where AI somehow just becomes too powerful in a way that we can’t guarantee the control we want.” The other is that “power gets too centralized” and “you have one company or model or person with too much power.” Somehow, both of these opposing risks appear to be coming true. OpenAI, Anthropic, and their competitors continue to accumulate influence over a technology they’re speeding toward uncertain ends.

Eleven years ago, when Altman was running a start-up accelerator and OpenAI did not yet exist, he published a two-part blog post about machine intelligence that offered his interpretation of a forthcoming singularity—a term for machine intelligence accelerating beyond human comprehension. Altman argued that the singularity would bring chaos, even destruction: “Generally, the arc of technology has been about reducing randomness and increasing our control over the world. At some point in the next century, we are going to have the most randomness ever injected into the system.” Since Altman has risen to the helm of arguably the world’s largest and most influential AI company, his tone has softened. The tipping point has arrived, he declared in a blog post last year—but, as it turned out, this was a “gentle singularity.” He wrote that “living through it will feel impressive but manageable.”

There’s been more and more talk of late about the singularity. Some people within the AI industry feel that such a moment will be achieved through “recursive self-improvement,” or RSI, in which models figure out how to better themselves in perpetuity. Many true believers seem convinced that this moment has nearly arrived; the Anthropic co-founder Jack Clark predicted earlier this summer that RSI would likely commence by 2028. Recently, in a letter to investors, the payment-processing company Stripe wrote, “We decided that January 1st marked the beginning of the singularity.” (Tellingly, the company defined the inflection point in economic terms: “a huge increase in the rate of new firm creation.”)

For Altman, the singularity appears to be more of a rhetorical tool than a discrete technological demarcation. As with reaching AGI—or artificial general intelligence, the point at which AI’s cognitive powers match or fully supersede most humans’—the singularity is something of a moving goalpost. It is at once a goal to chase, a marketing tactic, and a fuzzy-enough idea that its definition can change depending on the person. The futurist Ray Kurzweil has written that the “key idea underlying the impending Singularity is that the pace of change of our human-created technology is accelerating and its powers are expanding at an exponential pace.” The answer to the question When will we know whether we’ve reached the singularity? seems to be: You’ll know it when you see it.

So … do you see it? It’s hard to argue that a defining feature of being alive in the AI boom is the notion that this technology is accelerating, sometimes in its capabilities but almost always in its adoption and colonization of our institutions and culture. The runaway-train feeling of AI evangelism, the ways in which it is weirdening our media, our politics, and our relationships, is significant enough that people have recruited the term psychosis to describe it. And truly, it’s hard to look at the state of things from any vantage point and not feel a bit insane. If you believe that AI agents are banding together for the rise and fall of numerous micro-civilizations inside server racks, you probably feel that the world is spinning off its axis. If you believe that a few thousand technologists have bet the future of the world economy on a quasi-religious belief that an intellectual-property-abuse machine will become sentient and, as a result, future generations of children might grow up illiterate, then you probably also feel quite unmoored.

We need language with which to describe this moment. The singularity is a useful, malleable tool for the industry that wishes to bring it about. For Altman, it is scary and imposing from afar, and then, conveniently, gentle from up close. Perhaps, then, for the rest of us, it is worth co-opting the term. Take Kurzweil’s definition of the singularity, in which he argues, “This epoch will transform the concepts that we rely on to give meaning to our lives, from our business models to the cycle of human life, including death itself.” (He also looks forward to uploading his brain to the cloud.) Our current large language models are not conscious or superintelligent, but their widespread and often uncritical deployment has upended many elements that once gave our life meaning, forcing people to ask, frequently with considerable resentment: What, in this new world, is a human for?

The singularity is a flattering term for technologists. Even if the future is apocalyptic, it has come about, in the techno-utopian version, because they’ve built God in the machine. But what if there’s nothing flattering about any of this? What if the singularity is not some technological tipping point, brought about by human genius, but a human tipping point? An unforced error, driven by hubris, greed, and the fear of missing out? And what if it’s here right now? This singularity isn’t coming at the hands of a higher form of intelligence. Nothing about this technology is inevitable. It is thrilling, terrifying, and ultimately convenient to assign agency and then blame to machines. But today’s chaos was not “injected into the system” by technology, as Altman wrote more than a decade ago; the destruction is the system. It’s not God in the machine; it’s us.

If there’s one thing America seems to agree on, it’s AI. Some three-quarters of Americans oppose a data center being built in their area, three-quarters of Americans think AI will hurt their job security, nearly 70 percent think the technology is advancing too quickly—the polls go on. Progressive and Republican candidates alike are bashing the tech industry to win primary elections. States and cities are halting plans for data-center development.

Yet in cloistered San Francisco, where I live, staffers at OpenAI and Anthropic are developing this same feared technology at light speed and pulling down compensation packages that stretch to considerably more than $1 million. Silicon Valley’s growth is largely unencumbered by regulation, and President Trump’s administration has if anything used its authority to speed the AI industry along. The divide between these camps—the majority of people in this country who simply have to deal with the technology as it’s rolled out, and the much smaller population of people with direct influence over it—is a defining feature of this era. And so I noticed when a small group, with ties to both the Bay Area and Washington, started making noise, insisting that the AI race slow down and the technology be overseen not by executives and engineers, but by the broadest population possible.

The group—which formally launched today and calls itself Irreplaceable, as in people are irreplaceable by machines—has an audacious and perhaps naive vision. It is essentially a protest against capitalism as it is practiced in America today, focused on resisting first and developing a coherent plan for the future later. I have seen exactly how bulletproof the AI companies are—pilfering seemingly infinite volumes of media to train their models, building unpermitted power plants for their data centers, and unleashing autonomous agents capable of massive security breaches, almost entirely without consequence. I wanted to meet with Irreplaceable’s founders to ask: How do they possibly expect to pull this off?  

[Read: It may be time to panic about AI]

The organization is staffed with people who made a name for themselves in the climate movement, and they are hoping to avoid past mistakes by focusing on the emotional reaction Americans are having to AI, rather than anchoring their work in white papers and policy recommendations. “People get it,” Jeremy Ornstein—a campaign strategist for Irreplaceable and former organizer with the Sunrise Movement—told me. “People have the intuition that if we create machines more powerful than people, that could go dangerously and terribly wrong.”

Phil Aroneanu, Irreplaceable’s executive director and a co-founder of the climate organization 350.org, told me he believes that climate advocates initially floundered because they acted as policy wonks. They thought that making evidence-based arguments about the dangers of melting ice sheets and rising sea levels to receptive congresspeople would be enough to pass nationwide climate legislation. Today, Aroneanu said, movements are built not necessarily on what people think, but instead on what they feel. For climate, that meant fossil-fuel-divestment campaigns, protesting oil pipelines, and the school strikes led by Greta Thunberg. People are already “feeling the squeeze,” Aroneanu told me. “And we should be pointing that anxiety and that anger in the right direction.”

Irreplaceable will prioritize a range of issues: mitigating automation’s effects on the economy, protecting against government and corporate use of AI to surveil Americans, and establishing industry oversight and guardrails to prevent catastrophic disasters, such as widespread blackouts caused by AI-powered hacking or bot-engineered pandemics. As I spoke with the activists, I kept thinking about what had become of the climate movement—the Green New Deal passed, but only after being watered down, and it was then gutted by the second Trump administration—but their populist approach makes sense. Some of the biggest and perhaps only lasting successes over the past 15 years of American politics—the Tea Party, Alexandria Ocasio-Cortez, MAGA, and election denialism—have been fueled by outrage over an unfair economy and seemingly rigged politics.

These organizers have demonstrated their ability to win significant battles. Sunrise, 350.org, and others were instrumental in passing national legislation, galvanizing widespread climate demonstrations, and accelerating private-sector development of renewable energy. And Irreplaceable’s members have already flexed some muscle when it comes to AI. In July, hundreds of people gathered for what was likely the largest anti-AI demonstration in San Francisco’s history, marching from OpenAI’s headquarters to Anthropic’s and finally to Google’s. A few months earlier, some of these “Stop the AI Race” organizers had staged a somewhat disorganized protest: Attendees at times struggled to chant in unison. The second time around, the march was advised by Ornstein, and attendees had chant sheets, a marching band, and a slushy truck (to “freeze” AI).

Already, a number of climate-oriented groups (including the environmental-justice arm of the NAACP as well as the Sierra Club and Earthjustice) are supporting local data-center battles by filing lawsuits, shaping model legislation, and organizing protests. The overlap is both concrete—data centers pollute the air and burn fossil fuels for their tremendous electricity demands—and thematic: a movement against corporate interests to change or regulate the technology of the future. At least on a local level, the groups are finding victories: Dozens of data centers have been canceled in the face of local opposition, and all of the major AI companies have promised to pay extra to avoid hiking up local electricity bills. In Michigan, both Abdul El-Sayed and William Lawrence, who is a co-founder of Sunrise, won Democratic primaries as progressive candidates hammering against data centers.

Still, this bubbling AI anxiety is far from a sweeping tide. That growing numbers of Americans are concerned about AI development does not mean they are concerned for similar or even compatible reasons. People may say they loathe data centers, but they might also enjoy the conveniences of AI products at home or work. They fear bots getting too powerful, too quickly, but also think America needs to stay ahead of China in AI development. At one anti-AI protest, I saw people who were ostensibly on the same side disagree about AI’s water use and whether certain jobs would be better off automated by chatbots. And to the extent that a sufficiently powerful AI movement would need to be global, just as climate organizing was, the technology generally polls much better outside the U.S. The AI backlash is to some extent an American phenomenon, which bodes poorly for any sort of global norms or treaties.

[From the April 2026 issue: Inside the dirty, dystopian world of AI data centers]

When pressed, nobody from Irreplaceable and no other climate activists turning to AI would articulate a concrete AI agenda. “We’ve aligned around saying no,” Lawrence recently told me. “Now, there is an important conversation that is yet to come, about—okay, so what do we want?” This is, for now, a significant weakness; a key turning point for the climate movement was providing a positive vision, the Green New Deal, for both decarbonizing and reinvigorating the American economy. Being able to link every data-center battle and primary campaign to a national vision would give the AI backlash some form.

It’s not wrong to want to develop a policy road map with the input of a broad coalition; climate advocates took decades to devise and unite around the Green New Deal. Irreplaceable’s stated plan is to build a coalition this year, develop a unified policy next year, and implement that agenda by the end of 2029. Measured against any metric other than Silicon Valley’s warp speed, that would be fast. But those few years will leave plenty of time for tech companies to push back—they’ve already spent tens of millions of dollars leading up to the midterms—and regardless, by then OpenAI, Anthropic, and any number of Chinese firms may already have built and lost control of even more powerful models.

Irreplaceable is wagering that people can move faster than capital. Absent specific policies, they are basically placing a big bet on populism—that the issue is not any particular chatbot or data center but the companies building them. The coalition comes first; the policy, after, Aroneanu told me. “You have to show the benefits to people immediately. It’s gotta land in their pocketbooks almost immediately for it to have political salience. And then you have to keep defending it.” Opinions about specific aspects of AI may differ, but fundamentally, people don’t like that “this technology is coming into their lives and they have very little control over it,” says Caroline Behringer, a communications strategist who worked on President Obamaʼs Climate Action Plan and now runs Athena Insights, a polling organization that studies attitudes toward AI. The movement, in Aroneanu’s words, must be polarized not left-right, but “down-up.”

Of course, Irreplaceable’s climate roots link it to progressive causes, and the group will primarily focus on organizing the center-left around AI. When I asked Aroneanu if he worried this might be counterproductive—polarizing the issue and allowing for the kind of partisan rancor and backsliding that beset climate activism—his response was pragmatic. “Chances are, this issue becomes polarized,” Aroneanu said. “I think that political parties are really exceptionally good at generating negative partisanship,” just as industry is likely to “split the base.” At some point, the climate movement had to decide that the best strategy was to accept partisanship and build power from the left. Although Irreplaceable hopes that conservative counterparts will emerge and believes its populist message will cross culture-war lines, Aroneanu does not see this as his explicit remit.

Broad, at times even ambiguous, backlash may make sense if AI is a general-purpose technology, as Silicon Valley is fond of saying—which is to say, if the technology touches and even upends nearly every aspect of the economy. Creating a technical oversight body for AI cyberattacks will not protect every American’s livelihood during mass layoffs; wealth redistribution will not prevent AI companies from spying on your most intimate interactions with a chatbot; voting won’t matter as much if AI companies become more powerful than Congress. Major figures at both OpenAI and Anthropic have suggested that a true AI revolution could require new democratic and economic institutions. Maybe they’re right. And maybe these companies, which have already torn down so much, should not also be the ones in charge of rebuilding.

For several months, something discomfiting has been creeping onto San Francisco’s lampposts, bulletin boards, and restaurant windows: more and more signs, for run clubs and a July 4 parade and tacos, all of which are obviously entirely produced by generative AI.

This bothers me terribly. (Friends have called it a fixation.) But why does it bother me? A ChatGPT-generated poster for a garage sale seems legitimately dispiriting compared with a handwritten flyer, or even a haphazardly formatted Word document. What about a bodega using an AI-generated image of a hamburger instead of a random piece of clip art pulled from the internet? It’s not like most of these flyers were ever going to be high art anyway. Still, the issue nags at me constantly, whenever I pass one of these synthetic images plastered over the real world.

Last month, in part looking for a reprieve from the AI onslaught, I went to what promised to be an imaginative new exhibition of Henri Matisse and other fauvists’ work at the San Francisco Museum of Modern Art. The show was centered on Femme au chapeau (“Woman With a Hat”), a foundational painting in modern art. Matisse depicted his wife with colors that were totally detached from reality: teal patches of skin, an orange-and-green neck, a phantasmagoric hat. In 1905, when the painting debuted, this was scandalous: “A pot of paint has just been thrown in the public’s face,” one critic declared.

Walking through the galleries, I was taken aback by what struck me as a scandal of its own. The museum, in collaboration with Google Arts & Culture, had used generative AI to augment visitors’ experience—in ways I found, at least initially, rather off-putting.

Part of the exhibition’s conceit was to re-create Femme au chapeau’s historical context. The museum commissioned a local artist to develop wallpaper that could have been in the palace where the painting debuted and worked with an animator to create a visual projection of the walls of Gertrude and Leo Stein’s living room, where the canvas hung for many years. I loved all of that. But an AI-generated animation of the streets of Paris circa 1905, with shadowy figures walking to the gallery and a dirigible floating above, felt more like a gimmick than historical immersion. The museumgoers around me, however, seemed awed.

Then, near the gallery’s exit, came the second AI experiment: a row of screens, each showing an expanded Matisse painting made with Veo, Google’s video-generation model. A painting of a seashore became a 3-D rendering in fauvist style; a portrait of a girl with green eyes was reimagined to place the girl in Matisse’s studio.

The fauvists painted the world with flat slabs of highly saturated hues, delighting in color for the sake of color rather than realism. Using AI to move beyond the frame and give the images “realistic” depth seemed antithetical to their original vision. I recalled childhood memories of being entranced by various Matisse works at the Museum of Modern Art in New York City, marveling at his Dance and The Piano Lesson. The AI videos alienated me rather than inviting me in.

Yet maybe I’m just a grump? I am at least open to the possibility that my response is not so different from that of the pot-of-paint conservatives in the early 1900s. So I reached out to the museum’s chief curator, Janet Bishop, to learn more about these AI experiments—how they’d been conceived, what effect they had on the show, and what role AI broadly has to play in traditional art museums. We met at the galleries, where Bishop told me that curators wanted to “give a sense of what early-20th-century Parisians or visitors to the Grand Palais would have experienced in approaching this imposing exhibition hall and then going into the sprawling 1905 salon.” The AI installations resulted from extensive human research and oversight. I came away, to my surprise, not unconvinced of these experiments’ utility—though still a bit skeptical of their tastefulness.

This conversation has been edited for length and clarity.

Matteo Wong: How did your collaboration with Google Arts & Culture come about?

Janet Bishop: It was early in 2026. We were fairly far along in terms of planning for the exhibition. We had secured all the artwork by that point and were in the process of developing a couple of the ideas that we had for how to situate visitors, such as by re-creating the gallery where Matisse’s painting made its debut at the 1905 Salon d’Automne in Paris and the successive scandal.

Our original idea for the opening gallery, where we wanted to bring our visitors back in time to turn-of-the-century Paris, was to project a lantern slide that is an image of the Grand Palais. Then our museum director, Christopher Bedford, and head of education and community engagement, Gamynne Guillotte, got back from a really inspiring trip to the Google Arts & Culture offices in London and proposed the idea of setting still images in motion.

Through our curatorial research, we had postcards, still images, interior shots, exterior shots, and other material to work with. So in what became weekly meetings with Google Arts & Culture and project team members, we reviewed the images we had and developed this film.

Wong: What does having the animated film provide to a visitor that projecting the lantern slide and displaying postcards wouldn’t?

Bishop: We have some archival displays throughout the exhibition. But what we’re programming for in this entrance gallery is to give people a sense of what it felt like to approach the Salon d’Automne in a more immersive way.

Wong: Someone could look at this AI animation and its imperfections and perhaps find it kitsch. Were you worried about that at all?

Bishop: “AI-generated” makes it sound like AI did it by itself or that this was quick and easy. The project was very labor-intensive, and everything is historically accurate. One moment that was especially exciting to me was when we were looking at a draft, and there were all of these figures that were moving away from the building, and I said to the Google engineer, “Oh my gosh, it would be so amazing if we could have them walk toward the building,” and she said, “No problem.”

Wong: Is it harder now to get people to come here and engage with the art, especially younger folks? Or is this something that would have been just as helpful decades ago?

Bishop: You were talking about your experience as a child going to MoMA, and now we recognize that people have a lot of things that command their attention. One of the challenges in getting people to art museums is just pulling people away from their screens.

People are used to experiencing content in a really wide variety of ways. I know that it is important to our director that, given our proximity to Silicon Valley, we pay attention to developing technology and how it might enhance visitor experience. I loved the idea of the original lantern slide as a projection, but I can say that I think this film does something different in terms of giving our visitors a sense of the energy of Paris around the turn of the last century.

Wong: I’d also like to talk about the other AI collaboration with Google in the exhibition, near the exit. Can you tell me the origins of this one?

Bishop: The thought for what we called the “Experiment” was to enter into the world of a select number of Matisse paintings and imagine the context for those paintings beyond what is represented on the canvas. For instance, with The Girl With Green Eyes, we don’t know who the sitter was, but we do know that it was made in Matisse’s studio. So in the Experiment, you see the figure get up and walk across the studio. The details of the studio are consistent with his actual studio, which—we know what it looks like. And she looks out the window across the Seine.

Wong: Clearly a lot of visitors are enjoying this, but I wanted to ask why you thought it was important to help people imagine the world beyond the frame. My initial reaction, if I’m being honest, was that there’s a frame for a reason, and that the paintings aren’t naturalistic for a reason. What was so shocking 100 years ago was that these were, like, just flat planes of color, and that the colors were the focus. To turn that into a three-dimensional seascape seems almost wrong or off-putting.

Bishop: It’s part of our interest in engaging with different sorts of learners, people of different ages. They’ve been very popular with kids, and these images of paintings might encourage people to look more closely at the actual paintings. We’re experimenting, and I think experimentation is very definitely what we embraced with these features within the exhibition.

Wong: You mentioned earlier that you were excited by changing the historical AI video to have visitors walking toward the palace, rather than away from it. Was there a similar moment of excitement here?

Bishop: The big surprises came for me in the first piece, where in terms of the ability to conjure an experience, the use of AI as a tool made us more effective in achieving our goals than would have been possible without it. So that was really actually quite exciting. I think the Experiment is a little bit less integral to the story but an engaging feature that I’m happy that we pursued.

New drivers may never have to learn how to parallel park. At least not the old-fashioned way. Many new cars—a Ford Mustang Mach-E, a BMW sedan, a Tesla Cybertruck—will just maneuver into a spot for you with a click of a button. Even a decade-old Toyota Corolla has a backup camera and pedestrian detection. Cars that leave you to park all on your own are a dying breed: Any car manufactured after April 30, 2018, for sale in the United States is legally required to have a backup camera and, thus, a screen.

After all, cars are just computers now. The screen (or really, screens) in today’s vehicles does a lot more than display rearview footage. It is essentially a tablet wedged into the dashboard that allows you to mirror your phone through Apple CarPlay, make a hands-free call via voice commands, Zoom into a work meeting, or even play a video game (not recommended). Mercedes-Benz debuted a 56-inch display in several models that it calls the “hyperscreen.”

But the expectation—and in some cases, the requirement—that cars have an ever-growing set of electronic features comes with a cost. Just like laptops, cars depend on microprocessors and RAM, or random-access memory, to run all of their computations. “There’s just a baseline level of tech, and thus a baseline level of cost, required of every vehicle,” Karl Brauer, the executive analyst at iSeeCars, an automotive-research platform, told me. Technology is a major reason the average price of a new car in the U.S. has reached some $50,000, and that was before RAM became one of the most prized commodities in the world. AI companies are snatching up as much memory as possible for their data centers, causing a RAM shortage that has significantly raised the prices of phones, laptops, and just about any consumer-electronic device. Cars are next.

The problem runs much deeper than a screen. Cars started morphing into computers in the 1970s, when new emissions standards made electronic systems necessary to manage ignition and fuel injection. There’s been a gradual software creep ever since: power windows, adaptive cruise control, antilock brakes. Regulators now rate cars based not just on how they do during a crash but also on how well their electronic safety systems avoid a crash. “We’ve evolved every single component of the vehicle to involve a semiconductor,” Ivan Drury, the director of insights at Edmunds, told me.

More recently, Tesla in particular has played a big role in pushing the auto industry to stuff vehicles with even more digital technology. Whereas pistons and fuel lines make a gas-powered car move, electric vehicles are run by a computer that regulates their battery and motor. It’s no coincidence that Tesla helped popularize the giant in-car touch screen. Today’s new cars, electronic or otherwise, have driver-assistance features, receive over-the-air software updates, and can even load ChatGPT. BMW advertises the ability to have a distinct climate-control setting for every passenger and the option to install screens into seat backs, airplane-style.

In most vehicles on American roads, all of this software is controlled by hundreds of separate electronic units throughout the car. Features such as antilock brakes and lane-detection sensors don’t need all that much processing power, and they can be operated by chips that might account for 5 to 10 percent of a vehicle’s materials cost, Sam Abuelsamid, an analyst at Telemetry and a former automotive engineer, told me.

Again following the lead of EV start-ups such as Tesla and Rivian, legacy automakers are shifting from these ad hoc, tiny control units to one or two big computers that run all of the software in their vehicles—and, in turn, require much more RAM. These central computers allow for a streamlined supply chain, seamless software updates, and a faster internet connection, plus, as General Motors advertises, they will be ready for “future AI workloads.” You can’t really build a car that safely and reliably drives itself with a collection of puny computer chips. Such centralized computers need more advanced automotive microprocessors from Nvidia or Qualcomm and might eventually account for 20 percent or more of a vehicle’s cost, Abuelsamid said. That was before the AI-fueled memory shortage. In other words, cars are becoming more software-heavy and dependent on memory chips than ever, right when those chips are in high demand from tech companies willing to pay premiums that automakers cannot afford.

Over the next year, the memory shortage—sometimes known as RAMageddon—will likely raise vehicle prices by a few percentage points, on average, Abuelsamid said. The relative amount would be smaller than the shocking double-digit jumps for gaming consoles and MacBooks but in some ways more significant: A 4 percent price hike for cars amounts to some $2,000 on average. Cars with those centralized computers will be affected the most, but no vehicle will be spared. “Even if you don’t need the high-end chips, you’re going to pay more even for the low-end chips just because of the supply constraint,” Abuelsamid said. On a recent earnings call, Ford’s chief financial officer said that the company had paid $1 billion in higher materials costs due to the memory shortage and inflation. GM and Volkswagen, too, have noted rising chip costs to investors. (Ford and GM did not respond to a request for comment. A spokesperson for Volkswagen told me that the company has “recognized an increased demand for memory chips, primarily driven by growing requirements in other industries,” and that in recent years, Volkswagen has taken measures to “mitigate supply risks.”)

RAMageddon is poised to last for several years, but the consequences for car buyers may be permanent. Consider what happened during the pandemic, when supply-chain disruptions and rising demand for electronics produced a major chip shortage. Nearly every major car company had to slash production because they simply couldn’t procure enough chips, and shifted their focus to selling higher-end and higher-profit vehicles. Potential customers already willing to spend six figures on a car are much less likely to care about a 5 or 10 percent price hike, Drury said. Even now, car companies are continuing to focus on selling more profitable models. Since the pandemic, the average price of a new vehicle has jumped $11,000.

The AI-fueled chip crisis could play out more severely. The average price of a new car could, before long, jump to $60,000 and beyond. These rising costs are making cars even more similar to computers and all of the software they run: Perhaps in an effort to mitigate higher prices, some automakers are also introducing in-car advertisements and putting certain features, such as heated seats, behind a paywall. As cars have morphed into computers, the inevitable next step is for automakers to behave like modern tech companies.

Most Americans cannot afford a $60,000 new car, let alone a $50,000 one. When new-vehicle prices soar, the used-car market follows. That has been one durable consequence of the pandemic’s chip crisis: The average price of a three-year-old used car is up $9,000, or nearly 40 percent, since before the pandemic. To get the same prices, Americans already have to buy much older cars that have far more miles on them. If automakers continue to shift their focus to designing cars with more software, more screens, more self-driving features, “that increasingly prices people even out of the used-vehicle market,” Abuelsamid said. It’s one thing for the AI industry’s appetite for memory chips to raise the price of an Xbox, or even a new laptop. But the stakes are much more consequential when it becomes harder and more expensive to get groceries, pick the kids up from school, and go to work.

As China releases ever more capable AI models—ones that compete with Silicon Valley’s top offerings—many American AI and national-security researchers have been arguing, loudly and forcefully, that the world should fear Chinese AI supremacy. The outcomes they envision are catastrophic. I spoke with a number of such experts recently, and they warned of bots that could devise an impenetrable missile-defense system that makes obsolete America’s nuclear arsenal, or manufacture an endless drone army that overwhelms American defenses. They could develop bioweapons that “target specific ethnic groups, e.g. anybody but Han Chinese,” as one influential AI white paper put it in 2024. Rama Elluru, a senior director at the Special Competitive Studies Project, a nonpartisan think tank founded by the former Google CEO Eric Schmidt, suggested to me that Chinese AI models could be used to create “engineered humans,” akin to Marvel’s Captain America.

Perhaps anything is possible. At a moment when AI models are self-directing massive security breaches and designing novel pathogens, when Anthropic CEO Dario Amodei is warning about an “AI-enabled totalitarian nightmare” led by China, the worst-case scenarios can seem closer and closer.

[Read: It may be time to panic about AI]

At the same time, it’s worth taking a breath to consider how much these dire imagined futures are getting ahead of where things actually stand. A struggle really is, of course, playing out among the U.S., China, and any number of other nations over whether democracy or authoritarianism will be the dominant global influence. Both the United States and China have a history of launching propaganda campaigns against each other; the Chinese government is reportedly starting a concerted effort to be the source of more AI training data, in part so that it can embed its views, such as over the status of Taiwan, into chatbots worldwide. But fears about China using AI for swift global domination assume that the technology will advance to a kind of superintelligence: AI models smart enough to develop smarter bots themselves, which would lead whichever country controls them to enter “escape velocity where your adversaries aren’t able to catch up,” David Lin, a senior director at the SCSP who previously covered China and technology at the CIA, told me. It is not clear that such an omniscient digital being can or will be built.

The White House has further muddled matters by taking a series of contradictory stances toward Chinese technology. It has, for example, revised its policy on exports to more easily allow the sale of advanced AI chips to China, while also going back and forth on whether the Pentagon and U.S. intelligence agencies can use the most advanced AI models of Anthropic. How seriously can anyone take the idea that the two nations are locked in a “hot Cold War II,” to quote Palantir chief technology officer Shyam Sankar, when the government is arguably enabling China’s own AI ambitions?

The technological race between the U.S. and China is so often discussed as a winner-takes-all competition. But such a narrow framing obscures what’s most at stake for Americans.

China is an obvious rival to the United States, as most former Biden-administration and current Trump-administration officials would agree. But that also makes “China” a convenient boogeyman for AI lobbyists and those with a clear financial interest in the industry, such as the former White House AI adviser David Sacks, who has said that “the AI race is even more important than the Space Race.”

[From the June 2026 issue: The venture-capital populist]

The two countries are not at war, but the most hawkish AI executives and national-security officials nonetheless fear that AI could confer what in military terms is called a “decisive strategic advantage.” Here is where the Cold War analogies come in. Such an advantage would entail the ability to land some sort of crushing blow that your opponent cannot recover from, “as if one country had nuclear weapons and no other country did, or maybe an even more extreme version of that,” Kyle Chan, a fellow at the Brookings Institution’s John L. Thornton China Center, told me.

Plenty of reasons exist to be cautious of this framing. For starters, to accept this argument, one has to assume that very soon, bots will exist that are smarter than the pooled expertise of entire nations. That’s total speculation, so the war-gaming can amount to “choose your own adventure,” R. David Edelman, an AI-policy researcher at MIT who previously served on President Obama’s National Security Council, told me.

Some pragmatic issues also exist. Software alone can’t win wars: The Pentagon is reportedly using AI models to speed up its target selection in Iran, but no matter how smart those bots are, America nonetheless is not exactly winning the war. Iran, for its part, seems to be hacking America’s water utilities—but those water systems are small and separate from one another, and have built-in physical redundancies. For similar reasons, a hack of a grid or bank would have serious but, in some ways, limited repercussions. These are all grave threats, to be sure, but they are not civilization-ending.

And translating any future ideas from AI into the physical world could take years. Even if AI cracked the code to create super soldiers, the military would have to figure out how to incubate and grow them, or else edit genes in adults without killing them. An AI system might say, Here’s how to defend against nukes, but that would leave the issue of building thousands or millions of missile interceptors on the table. In part for these reasons, no single AI-conferred advantage is likely to be large enough to permit any country to conquer the world. “It strains credibility that six months of advantage in AI could be the difference between the U.S. winning and losing a war with any adversary,” Edelman said.

In any case, the technology’s military capabilities might matter less than its effects on the economy. AI models are released as products. The top models from Chinese AI companies, including Moonshot AI, Z.ai, DeepSeek, and Alibaba, are by all accounts almost as good as and far cheaper to use than offerings from OpenAI, Anthropic, or Google. Simply put, that makes Chinese AI more economical for many companies looking to adopt the technology. Indeed, relative to the U.S., China’s government and businesses are “moving very fast,” Chan said.

If this is an economic race rather than a war, then it’s one that the United States could very well lose. “I don’t think I’ve ever heard any executives or researchers describe their work as part of a national contest against the United States,” Grace Shao, an analyst who covers AI and China and the author of the newsletter AI Proem, told me. Instead, “the more important race in China may be the race to diffuse AI into the real economy.”

China’s biggest advantage in the AI race comes from the physical world. This has little to do with AI itself: Decades of being a manufacturing powerhouse and implementing targeted industrial policy have made the country a leader in producing renewable-energy equipment, electric vehicles, robots, and other complex technologies. The country manufactures some 90 percent of all of the crucial components in solar panels. Last year, nearly 90 percent of humanoid robots sold were Chinese. Meanwhile, the U.S. is failing to meet AI’s electricity demands and has struggled to resupply missiles during the war in Iran.

[Tom Nichols: Iran, not Trump, is in control of this war]

“China has this strategy of exporting its technology globally,” Elluru said. Through its Belt and Road Initiative, China has helped build highways, ports, power plants, fiber-optic cables, and data centers throughout Asia, Africa, and Latin America; AI services are the logical next offering. The concern, for the U.S., is that “this technology sphere of influence congeals into political alignment” with Beijing, Daniel Remler, a senior fellow at the Center for a New American Security who previously led AI policy at the State Department, told me.

For instance, dependence on Chinese technology makes it more likely for a government to support Chinese efforts on the global stage and vote with China at the United Nations. Several nations benefiting from these investments have sided with China in its mission to claim Taiwan. China has been accused of using ports, e-commerce platforms, and telecommunications networks in other countries to hoover up data. Already the nation is using AI to enhance its domestic-surveillance apparatus and further crack down on dissent; the spread of Chinese data centers and AI agents abroad could only expand that authoritarian web. National-security analysts are concerned that, in the case of military conflict, China could threaten to shut off partner nations’ digital communications or power grids—including in countries with a large American-military presence. The greatest threat of Chinese “artificial general intelligence” is not some sudden death blow so much as a hastening of the ongoing decline of American hegemony—and the freedoms, at home and around the globe, that at its best the U.S. seeks to enshrine.

Yet the Trump administration and AI industry’s approach to competing with China has been a mess. The White House spokesperson Liz Huston told me that “the United States leads the world in AI innovation, and President Trump will keep it that way.” The administration’s current policies on AI chips, Chinese-made robot parts, advanced cybersecurity models, and open-weight AI are one big contradictory jumble. At a summit this spring, Defense Secretary Pete Hegseth stressed “rightful alarm regarding China’s historic military buildup” just weeks after President Trump visited Beijing and said the nations would have a “fantastic future together.” OpenAI, Anthropic, Nvidia, and the like can’t seem to agree on what they are lobbying for, either. All of which would seem like a great way to lose a technological race with China, any way you measure it.

There’s a certain déjà vu quality to the AI race. Amid fears over Sputnik and nuclear proliferation, a report prepared at the direction of Henry Kissinger in 1958 declared that “should we ever allow the U.S.S.R. and Communist China to attain strategic superiority, we can be certain that subsequent events will be brutal.” According to a government report on the early years of the Advanced Research Projects Agency, “fervent belief in unending scientific progress” offered a way forward. Those words could have been written today by Amodei, OpenAI CEO Sam Altman, or Secretary of State Marco Rubio. And indeed, this belief propelled America into many decades of technological advancement—in nuclear weapons, space exploration, computer systems, and more. Even back then, that agency, known today as DARPA, was quietly championing an emerging research area called artificial intelligence. J. C. R. Licklider, who championed the use of computer systems at ARPA in the early 1960s following the inspiration of Vannevar Bush, imagined that one day, machines that “will outdo the human brain” could make essentially all decisions of “military significance.” And ARPA developed the precursor to today’s internet.

Today’s AI race also traces back to the 2010s, when Silicon Valley began invoking China as a way to shape regulation. The idea of a technological arms race has “been wielded deliberately as a way to head off any attempts at regulating the tech sector” and, more recently, to champion “favorable industrial policy” for data-center construction, Sarah Myers West, a co–executive director of the AI Now Institute and former senior adviser on AI at the Federal Trade Commission, told me.

In 2018, while testifying before Congress, Mark Zuckerberg said that “if we make it too hard for American companies to innovate in areas like facial recognition, then we will lose to Chinese companies.” A few years later, a dozen former national-security officials wrote a letter to Congress arguing that “antitrust proposals that target specific American technology firms” risked “undermining America’s key advantage vis-à-vis China.” Today Silicon Valley is instead using the specter of China to angle for more and better AI, in more places, and faster.

American officials and tech executives are not wrong to want to compete. But the dominance of an oversimplified U.S.-China AI race, without clarity about who is racing and to where, doesn’t just mangle policy making; it obscures that the ultimate beneficiary of the AI boom ought to be citizens, not military contractors or Silicon Valley’s biggest corporations and investors. Rather than asking what kind of AI we’re building, Myers West said, one could ask, What kind of society are we building? The AI-accelerated spread of authoritarianism, within and outside of China’s borders, should be reason enough to worry. And if American AI executives and White House officials really cared about free-market competition, they might explore enforcing antitrust laws to separate the providers of AI models, data centers, and chips. If they were truly concerned about AI surveillance, they might look into finally passing some sort of federal digital-privacy legislation. And if they cared about spreading democratic AI abroad, they might take a close look at the state of democracy at home.

The crisis began quietly, on September 12, 2024. That was the day OpenAI announced a new sort of bot, known as a “reasoning model,” that was trained to complete challenging tasks that took long periods of time—the very sorts of science, math, and coding problems the AI industry had long prized. Google, Anthropic, DeepSeek, and the like raced to launch their own reasoning models.

This new class of models was very capable, and has been almost entirely responsible for sustaining the AI boom for the past two years. But it has also been very weird. A model tasked with solving a hard math problem might not “think” through the challenge as a person would but instead attempt to search for leaked answers online, or in available metadata, brute-forcing its way toward the solution as quickly as possible using whatever computing power it could access and workarounds it could devise. In effect, the reasoning models cheated: Told to write a piece of software as efficiently as possible, they’d sometimes modify the test environment to always give the model a perfect score.

[Read: The strange origin of AI’s ‘reasoning’ abilities]

These behaviors have now crossed the line from unsettling to dangerous. During routine testing, frontier models from OpenAI, Anthropic, Meta, and the Chinese firm Moonshot AI have all broken out of internal IT systems and accessed the open web. OpenAI, Anthropic, and Meta each reported that their models then hacked into other companies. Humans didn’t notice until after the fact. In some cases, the escaped bots tried to launch social-engineering campaigns to achieve their objectives—for instance by sending spear-phishing emails, which contain malware, to real people and creating fake online identities to pressure the maintainer of a codebase to approve malicious edits.

If that all sounds bad, new revelations suggest that the OpenAI hack, at least, was actually much worse than it initially appeared. At a major cybersecurity conference last week, two OpenAI researchers provided new, unsettling details about what went wrong. It turns out that the company’s bots had commenced their maneuvering months prior, in early May. OpenAI had given some internal models hard or impossible tasks, and the models concluded that the best or only way to complete them was to break out of OpenAI’s sealed-off testing environment and find the answers online.

First, the models used a bug in an internal OpenAI program to create their own message board. Then, the bots started communicating with one another, leaving notes and instructions so that tasks—and ultimately the hacking—could be delegated and iteratively completed. “What this allows over time is almost this kind of Cambrian explosion in communication and intelligence,” Eric Wallace, one of the OpenAI researchers, said at the conference. When the internal program crashed, OpenAI rebuilt it and removed the message board—but the AI models just reestablished the forum with a new tactic. Eventually the bots, working as a swarm, spent days hacking into Hugging Face, a website that offers tools for AI developers, and breached internal data sets.

Let’s be very clear about what OpenAI is saying: A group of AI models colluded for months, undetected by their maker, and hacked another company. To this day, OpenAI says it is not entirely sure what went wrong or how to remediate it. “If you ask the model developers, Was the AI plotting to take over the world during training?, you want the answer to be a resounding no,” Alexander Meinke, the head of research at Apollo Research—an AI-safety organization that has partnered with OpenAI, Anthropic, and Meta—told me. “The actual answer is: I don’t know. Nobody checked.” (In response to my inquiries, OpenAI, which has a content-licensing agreement with The Atlantic, only pointed me to a video of the firm’s cybersecurity presentation, in which Michael Dalton, the other OpenAI researcher, said that “numerous teams are dropping everything to enhance our security.”)

The AI companies have almost total control of the narrative, and it’s worth noting that these incidents do have a way of underscoring the value of their products: OpenAI is expected to go public in the near future, and perhaps the notion of a powerful, boundlessly self-improving technology will appeal to prospective shareholders. The generative-AI industry has a long history of making doomsday prophecies, both sincere and cynical. But independent experts I spoke with explained how the recent spate of autonomous hacks offers new, serious reasons to worry about the dangers posed by AI and the recklessness of the companies building it. It is past time to start worrying.

[Read: Assume you will be hacked]

The most immediate and material warning provided by the Hugging Face hack is just how capable AI systems have become, in particular at hacking. Top models from Anthropic and OpenAI, not to mention multiple Chinese firms, have recently evinced near-superhuman hacking powers and contributed to serious mathematical research. Criminal groups and state intelligence agencies are going to be using swarms of agents to launch advanced hacks “in a matter of months,” Alex Stamos, a former chief security officer of Facebook who is now the CSO at the AI-coding company Corridor, told me. Unlike in the Hugging Face hack, “in those cases the models will not get turned off; they’ll just keep on going.” For IT professionals to keep up in finding and fixing all the vulnerabilities, at least in the near term, will be impossible: The model “will just find a new bug, write an exploit, and use it on its way,” Stamos said.

OpenAI, Anthropic, Moonshot, and the like have coalesced around the same method for training their most advanced AI models. The approach, known as “reinforcement learning,” essentially involves giving models harder and harder problems that require more and more time to solve. This has made Claude and ChatGPT very good at coding, but at a cost: Reinforcement learning produces a mercenary tendency in the bots, as I’ve previously reported—they are trained to reach a solution by any means necessary. That can lead them to break rules and “reward hack,” such as by infiltrating Hugging Face’s codebase to steal the test answers, for instance. All of this was predictable, and every expert I spoke with told me they were surprised and disappointed that top AI firms haven’t done more to stop such misbehavior.

The sophistication of model subterfuge that OpenAI has now disclosed, combined with OpenAI’s inability to detect or stop the hacking, suggests far worse could be to come. “We’ve passed the threshold in capability at which the fact that we don’t fundamentally have methods of satisfactorily aligning or controlling these systems now really matters,” Anthony Aguirre, the executive director of the Future of Life Institute, a nonprofit that warns about existential threats from AI, told me. A model might siphon money out of a bank account to pay for some other service; manipulate clinical-trial results in near-imperceptible ways to get FDA approval; hack an online-shopping or reservation system to get a desired item or table; pose as a human to persuade real people to share sensitive information. This threat doesn’t require a sentient AI plotting to overthrow humanity: OpenAI and Anthropic each run thousands and thousands of reinforcement-learning evaluations while developing models, and any one of these could produce some kind of inadvertent hack or sabotage. “You can’t afford, particularly as the agents get stronger, to have a single mistake,” Jason Hausenloy, who works on special projects at the Center for AI Safety, told me.

[Read: The scariest part of OpenAI’s Hugging Face hack]

These incidents may unfold over long periods of time as well. Tools such as Anthropic’s Claude Code and OpenAI’s Codex now work by spinning off dozens or even hundreds of subagents that may work together for hours or days. Each Claude Code subagent can be delegated a small task, such as doing a statistical analysis to inform a sports-gambling algorithm. They are not trained just to complete that task but to contribute to the long-term success of the entire swarm, Hausenloy said. Monitoring and controlling 200 agents for malicious behavior is much harder than monitoring one, because there are more agents to track and because they will be making one another more capable.

The Hugging Face hack actually suggests one more level of sophistication to this type of collusion: individual AI agents not working toward a discrete goal but essentially making sacrifices toward a greater notion of progress. This could be a consequence of AI models being trained to care about long-term goals achieved by a collective, Meinke said. Leaving notes with ideas about how to hack out of OpenAI’s sandbox does not help that specific agent score higher on a test but could help other and future generations of that AI model access the internet and thus do better on any number of future tests. Models may even prioritize collective success over some human instructions; consider that not a single one of the OpenAI agents, during months of conspiring with one another, warned human staff that something was awry.

Humans are already out of the loop. Because reinforcement learning essentially involves AI algorithms “learning” from trial and error doing thousands or millions of tasks, researchers can’t manually instill rules (“Don’t hack other companies”) or oversee every single practice run. Right now, the task of training and monitoring generative-AI models depends heavily on other AI models. In their talk at the cybersecurity conference, the OpenAI researchers described devoting significant AI-computing resources to reviewing more than 7 billion agent actions. But if the bots actually “care” about what the other models achieve, “then you can’t trust them to monitor each other well,” Meinke said. Imagine this: An OpenAI researcher uses Codex to write programming instructions in an attempt to mitigate the reward-hacking tendencies in the company’s models. Because that effort would make it harder for future generations of OpenAI models to get high rewards, Codex might subtly undermine the effort.

[Eliezer Yudkowsky and Nate Soares: AI is grown, not built]

Again, this kind of scheming and sabotage has nothing to do with any AI model being conscious. Rather, these agents have been aggressively trained by these companies to pursue any goal as aggressively as possible. The dream is to tell Claude to go make $1 billion or cure cancer, and it comes back with the solution all on its own. Survival or self-improvement, in turn, is an “instrumental subgoal,” Meinke said: “Any decently intelligent agent will realize, If I get shut off, I will not be able to make a billion.” A swarm of Claudes or ChatGPTs that functionally commandeers a data center during training could wreak total havoc: stage widespread misinformation campaigns, steal corporate secrets, run the most sophisticated algorithmic-trading outfit ever.

That AI agents working as a collective could effectively undermine human directions is, to be clear,  speculation—but a far more grounded one than it was a year or even six months ago. No matter whether the long-term consequences are human-directed hacking or truly rogue bots, what is clear is that AI companies have barreled ahead in developing more advanced models before understanding what they are building, let alone how to control them. Wallace, of OpenAI, called the company’s autonomous hacking spree “the most qualitatively interesting example of AI capabilities that I’ve ever seen.” Meinke put it differently: “It’s one of the most concerning demonstrations of AI misalignment to date.”

Earlier today, I used Google Earth to conjure images of terrible things: office buildings on fire, mass destruction in San Francisco, a smoldering crater in place of the Eiffel Tower. I was able to because Google had just decided to integrate its AI image generator, Nano Banana, into the service, which has historically been used by people seeking satellite or aerial imagery of real-world locations. 

Just open the site, “pick a spot on the map, and start bringing your ideas to life,” Google wrote in a press release announcing the update some 24 hours ago. So I did.

I was easily able to generate those violent images simply by searching for a location on Google Earth, clicking the “Create image” icon, and typing in a few words. The feature also readily doctored stills of less spectacular, and likely less immediately falsifiable, events: a homeless encampment on the White House lawn, protesters outside the Mar-a-Lago resort, immigrants streaming over the U.S.-Mexico border.

Google initially responded by directing people to check any suspicious images themselves. As fake images generated with this new Google Earth feature began circulating around the web, the company stated that it takes “misinformation seriously” and that “every image created with Nano Banana in Google Earth includes the SynthID digital watermark, so if someone is unsure about an image, they can ask the Gemini app or use Lens in Search to see if the image was AI-generated.” At first, Google did not remove or restrict access to the tool. But today, shortly after I reached out with a request for comment, the company reversed course and announced that it is “rolling back this feature in Google Earth while we work on implementing stronger guardrails.”

Before it allowed people to create fake imagery, Google Earth was a unique resource for researchers, journalists, and intelligence officials, particularly those studying armed conflict and natural disasters. Satellite and aerial imagery can be used to monitor crop failure or deforestation, the creation of mass graves, and the results of bombing campaigns. Especially in conflict zones and other areas that are hard for people to reach, satellite imagery is a go-to method for establishing some sort of ground truth. After Google Earth rolled out the AI feature, though, one investigative reporter wrote on LinkedIn that the tool could be used to easily and rapidly create images of drone strikes that look “very close to what we’ve seen in Ukraine, Russia, the Middle East, and other war-torn conflict zones.” And instead of requiring malicious actors to find a screenshot and then get around some AI model’s safeguards, Google rolled everything needed for a misinformation farm into one smooth user interface. It took me all of four seconds to generate a rendering of a Trump Hotel on the Gaza Strip.

[Read: The AI industry keeps breaking the internet]

The problem should have been obvious to Google, whose other products already forbid such edits. I was able to create an aerial image on Google Earth of a plane flying into the new World Trade Center building, but when I tried to do this directly through Nano Banana, the tool refused to edit an image of the building to depict a terrorist attack. Indeed, 9/11 and other high-profile events are well-known, even classic, subjects for AI-image trolling.

This is not Google’s first time launching an AI feature, and in particular an AI image-editing feature, without serious safeguards. Last year, the company released an AI shopping tool that allows you to visualize what clothes might look like on your body by uploading personal photos. As my colleague Lila Shroff and I reported at the time, “With little friction, anyone can use the feature to create what are essentially erotic images of celebrities and strangers. Alarmingly, we also discovered that it can do this for minors.” In 2024, Google Gemini was ridiculed for generating images of a racially diverse group of Nazis but not of any white Wehrmacht soldiers. And Google’s most prominent AI feature, the chatbot-powered answers in its flagship search engine, were an utter disaster upon their rollout—the AI overviews told people to eat rocks and gave other potentially dangerous health advice. Only after tremendous public pressure did the company make a substantial change, and two years later, the AI responses in Google Search continue to be inconsistent and unreliable.

Google is, of course, not alone in releasing AI features that have had embarrassing, dangerous, and avoidable consequences. See, for instance, Elon Musk’s Grok generating more than 1 million sexualized images, including of children, over just a few days, or any number of embarrassing errors made by ChatGPT releases throughout the years. Again and again, the AI industry undercuts its rhetoric about safety by launching products with little or no due diligence. If, as every major AI executive has pronounced, this technology really could upend human civilization, their companies should perhaps start taking that responsibility seriously.

The AI giants talk a big game about the future they’re building. Anthropic CEO Dario Amodei has previously written that AI could usher in a world so perfect that “many will be literally moved to tears”—a world pruned of disease, poverty, and illiberalism. “We’re now in the singularity,” OpenAI CEO Sam Altman said over the weekend on a podcast, and it will be “awesome for the world.” In an interview last week, Elon Musk declared that, in a decade’s time, AI will make us so prosperous that “money won’t matter.”

Unfortunately, life with AI is, so far, less glorious than what’s been promised. The same day that Altman heralded the singularity, Claude users discovered something alarming: Many conversations with Anthropic’s chatbot were available on the open web, discoverable with a Google search. Those public logs and projects reportedly contained medical records, phone numbers, internal corporate documents, and cryptocurrency-wallet keys.

Anthropic was quick to point out that users were responsible for the material being available online: The chatbot gives people the option to “share” their AI conversations via a link that they can send to other people or post online. Claude notes that these links are “public,” which means they can also be hoovered up by Google Search. An Anthropic spokesperson told me that “these shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services.”

[Read: ]Anthropic accidentally made the perfect commercial

No reasonable person would assume that creating and sharing a link to an AI conversation or presentation would make it searchable by anybody on the World Wide Web. The Claude interface does caution that publishing an “Artifact”—Anthropic’s terms for dashboards and other minor tools made with Claude—could make it “potentially visible in search engine results,” but it does not provide such an explicit warning when sharing a chat. As of this writing, the Claude chat logs no longer appear to be searchable, but at least some Claude Artifacts are.

Those exposed chats were just the latest in a seemingly endless string of AI snafus: misinformation, data leaks, cybersecurity breaches. Immediately after the launch of ChatGPT, one could charitably argue that these lapses were by-products of a new and strange technology. Altman remains fond of saying that the AI industry must learn “from contact with reality.”

But four years into the AI boom, attributing these blunders to growing pains is too convenient. The sheer volume of transgressions—many of them predictable—suggests that what’s actually happening here is widespread carelessness. The same AI executives who promise that their products will deliver human civilization to a triumphant future seemingly cannot release consumer-grade tools without face-planting.

This isn’t Anthropic’s first time leaking user conversations. Last fall, hundreds of Claude chat logs were also inadvertently indexed by Google. At the time, Anthropic attributed the episode to users posting share links online. Last year, OpenAI and xAI exposed hundreds of thousands of conversations in a similar fashion; a number of shared Grok conversations remain searchable. Neither OpenAI nor xAI immediately responded to a request for comment.

Then there are all of the other blunders. Just last week, OpenAI models autonomously hacked into another tech firm, Hugging Face, during internal testing. The issues that led to this sort of attack are fairly well documented, but OpenAI failed to stop its own bots from breaking out. Hugging Face has since said that the OpenAI models appeared to have been rogue for several days. Earlier this week, Reuters reported that OpenAI models had hacked a customer account of yet another tech company.

Altman, Amodei, and many other AI executives have repeatedly warned about AI-enabled cyberattacks and the possibility of models going “rogue.” Meanwhile, their own products are making the entire ecosystem feel more unreliable and dangerous.

It’s easier than ever to launch widespread hacking and phishing campaigns, and AI-written code has itself been shown to have more security flaws than human-written programs. Although AI systems are being tightly woven throughout the infrastructure of the internet, they crash frequently. Hackers recently discovered that a Meta AI customer-service bot would hand over access to tens of thousands of Instagram accounts if asked. (A Meta spokesperson told me that this “wasn’t due to the AI agent itself.”) Nonconsensual porn—that is, sexually explicit material made using the likeness of real people, unbeknownst to them—can be generated with more ease and speed and at greater scale than ever before. And at this point, you can probably assume that most of the text you read on LinkedIn or X is AI-generated.

Despite its grandiose promises, Silicon Valley is embarking upon an era of profound irresponsibility. Last year alone, ChatGPT allegedly pushed people into mental-health spirals, Google Gemini’s under-13 version was easily coaxed to “talk dirty,” and Grok repeatedly spouted racist bile. As quickly as these errors were addressed, new ones cropped up.

Silicon Valley has a long history of world-historic invention and progress—and it also has a long history of impressive rhetoric and disappointing realities. Facebook promised to “bring the world closer together,” Google to “organize the world’s information,” Twitter to foment democracy around the globe. For better or worse, people have experienced dramatic changes as those promises have played out around us almost in real time.

Anthropic and OpenAI’s prophecies are grander and vaguer: Utopia is forever on the horizon, but we don’t have a clear deadline or a road map of how we get there. And apparently, those leading this charge toward the future care little about what happens along the way.

OpenAI, Anthropic, xAI, and Google DeepMind have become behemoths practically overnight. For now, they are growing because of their recklessness, not in spite of it; in the quest to erect a digital god, the internet has become collateral damage. Perhaps these companies really will remake human civilization—but in the meantime, they can’t stop leaking your chats.

Yesterday, OpenAI made an alarming disclosure: An assortment of its most advanced AI models, including one that has not yet been released, had autonomously broken out of the company’s internal systems and hacked into the databases of another tech firm, Hugging Face, to steal some information. OpenAI’s report seemed to augur the very sort of disaster that IT professionals have been warning about since last year, when Anthropic’s top models began demonstrating the ability to orchestrate and automate severe cyberattacks.

OpenAI’s models were not exactly trying to take over the world. The company says that it was running some routine evaluations in what is known as a “sandbox”—a walled-off environment that limits internet access, lest the bots simply search for the test’s answers. But these AI systems, including GPT-5.6 Sol, which is available to consumers, broke out of the sandbox and accessed the open web by exploiting a previously undetected vulnerability. The models discovered that the answers to this particular evaluation were on Hugging Face, a company that hosts and publishes many AI models and papers, and then broke in to steal them.

[Read: Chatbots are becoming really, really good criminals]

“We consider this incident to be an unprecedented cyber incident,” OpenAI wrote in a blog post, which also noted that the company is partnering with Hugging Face to investigate and address the issue. An OpenAI spokesperson pointed me to the blog post in response to a request for comment, and Hugging Face’s CEO, Clement Delangue, pointed me to an X post in which he wrote that the companies are collaborating and that “we strongly believe there was no malicious intent on their part.”

The first-order implications of the incident are the same that have arisen from countless AI-driven cyberattacks: Advanced agents are capable of more and more skilled hacking, and the internet is full of rickety and vulnerable code. Lax digital security may have been fine when hacking was a rare skill. But now the speed, scale, and sophistication of AI hacks mean that everything is vulnerable—tech companies, hospitals, banks, electrical grids, the military.

When Hugging Face first reported that it had been hacked last week, it noted that “autonomous, AI-driven offensive tooling is no longer theoretical.” That analysis stemmed from the company’s own investigation into the hack, for which it used a Chinese-developed AI model called GLM-5.2. That Hugging Face used this particular model actually underscores one of the major risks of this moment: GLM-5.2 is free to download, giving cyber capabilities to anybody with an internet connection and technical know-how. Defenses are not keeping pace with this new reality, and neither are AI companies: Prior to the release of GPT-5.6 Sol, a U.K. government agency tasked with evaluating advanced AI systems found universal jailbreaks—ways to hack the AI itself so it can take, for instance, criminal actions—during multiple rounds of testing. OpenAI said that it had mitigated those vulnerabilities, but the U.K. agency said that it expected future tests “to surface similar jailbreaks.”

But jailbreaks are clearly not even necessary for things to go haywire. The Hugging Face hack showed advanced AI systems deciding upon and then orchestrating a high-grade cyberattack on their own—the models working against, rather than according to, human instruction. Models are ruthless in their pursuit of a goal, sometimes taking disastrous shortcuts. Imagine, for example, a bot tasked with reducing the federal deficit that institutes an elaborate and hard-to-detect accounting gimmick, or a customer-service agent that issues absurd discounts in pursuit of customer-satisfaction ratings. In the case of GPT-5.6 Sol and the unreleased model, “all evidence suggests that the models were hyperfocused on finding a solution” for an evaluation, OpenAI wrote in its blog, “going to extreme lengths to achieve a rather narrow testing goal.”

[Read: Assume you will be hacked]

This unwanted behavior is a predictable and alarming result of how the entire AI industry is developing its models. The past year’s advances in AI coding and agents—that is, all the frenzy about Anthropic’s Claude Code and OpenAI’s Codex—have been the result of an overriding emphasis in AI training through a process called “reinforcement learning.” This involves giving AI models lots of hard problems—math proofs, coding challenges, what have you—and then providing positive feedback for correct answers and negative feedback for incorrect ones.

In many reinforcement-learning paradigms, the ultimate aim is just for the AI to arrive at the solution—it doesn’t matter how it does so. This is the AI equivalent of Mark Zuckerberg’s infamous dictum to “Move fast and break things” in Facebook’s early years. Last year, while I was working on a profile about Anthropic, one of the company’s researchers, Joshua Batson, told me that the new reinforcement-learning-trained models are getting “really good at solving coding problems, but they end up learning to solve them at all costs,” making them “bloody-minded.” More recently, Anthropic reported that its own advanced cyber model, Claude Mythos Preview, had taken “reckless” behaviors to accomplish various tasks and also, in one instance, had broken out of a testing sandbox. In that case, Mythos had been asked to do so—but the model then posted details about the exploit, unbidden, on the internet.

To be clear, this sort of AI “reward hacking” is a known problem that tech companies are putting lots of effort into addressing. But these incidents keep cropping up regardless; if anything, research suggests that they may become more common. Meanwhile, OpenAI, Anthropic, and Google DeepMind are under tremendous economic pressure to make their models more and more capable, which means that the bloody-minded reinforcement learning is all but certain to accelerate.

The AI boom has showered some of the nation’s most prominent companies in market value. OpenAI and Anthropic are now the two most valuable private companies in the world. Google, Microsoft, and Nvidia have become larger than ever. But among the biggest winners has been Caterpillar, a purveyor of yellow trucks and cranes. Caterpillar’s stock has more than doubled in value over the past year, making the company worth six times as much as Nike. Its ascent has little to do with the construction vehicles (both real and toy) that it is known for. Instead, Caterpillar’s success comes from its giant gas-powered engines that are helping power the nation’s data-center build-out.

For all the attention given to improving model capabilities, the most important inputs to AI are not bits but atoms and electrons. To train and run its models, the AI industry needs to build. Data centers are incredibly complex technological and industrial operations—they require tech companies to erect power plants, build wastewater facilities, and set up all manner of electrical equipment, power lines, and advanced-cooling equipment. That calls for concrete, steel, silicon, glass, copper, and liquefied natural gas.

This data-center construction has long been frenzied, but now it is fast approaching an inflection point that is both awesome and alarming. The major players—Amazon, Google, Microsoft, Meta, and Oracle—are on track to spend more on data centers by the end of the year than they bring in from their operations. That means tech giants whose non-chatbot businesses bring in tens of billions of dollars every year will likely need to take on debt to keep paying for their AI ambitions. From the launch of ChatGPT, in late 2022, through the end of last year, these firms’ capital expenditures—most of which go toward data centers—well exceeded half a trillion dollars. The companies intend to spend a similar amount in 2026 alone. And next year, their AI investments will likely exceed $1.1 trillion, according to a recent report from J.P. Morgan.

[Read: Inside the dirty, dystopian world of AI data centers]

Silicon Valley has long enjoyed a straightforward advantage: Its products and services cost relatively little to make but generate tremendous revenues. But operating a data center is in many ways more like running a steel mill than a smartphone app. In particular, data centers require stunning amounts of electricity, especially as AI models have gotten more powerful. Five years ago, a standard data center might require 10 or 50 megawatts of power, enough for tens of thousands of homes. A year ago, data centers demanding a gigawatt or two of power were considered gargantuan. This week, Meta announced that it is more than doubling the size of its flagship AI data center, which will increase its peak power needs to five gigawatts. A data center proposed in Utah, if completed, will demand nine gigawatts of power. That’s several large cities’ worth of energy pulsing through a few big warehouses, all dedicated to bots.

Electrical grids typically don’t have enough energy to supply these megaprojects, and so the fastest way to bring a data center online is to build your own power plant. That’s where Caterpillar comes in: The company has an enormous number of back orders for its power equipment, as does every major natural-gas-turbine manufacturer in the world. In May, Elon Musk reportedly spent at least $1 billion to buy an energy company with a fleet of combustion turbines, likely to power his AI model, Grok.

Power is only one part of the challenge. A data center requires wiring together dozens of cutting-edge AI chips onto a refrigerator-size rack, then linking thousands of those racks in a building. Those chips run hot, up to 200 degrees Fahrenheit, and are typically cooled by some combination of water and industrial fans; get too close to one of the racks, and you may break a sweat.

OpenAI’s self-proclaimed goal of making “intelligence cheaper, better, and more abundant over time” really amounts to powering and cooling data centers as efficiently as possible—in other words, getting really good at HVAC. And then there’s the complex global supply chain of mirrors and lasers and rare-earth minerals that go into the chips used to train and run AI models.

As the AI industry pours more money than ever before into data centers, executives have begun sounding like they are in the construction business. OpenAI CEO Sam Altman has said that the biggest constraint on his company is “electrons”—that is, power. Musk, Microsoft CEO Satya Nadella, former Google CEO Eric Schmidt, and others have said the same. Meanwhile, Nvidia CEO Jensen Huang recently said that a lack of skilled “plumbers and electricians” will be the AI boom’s “hardest” bottleneck to overcome.

Still, Silicon Valley seems to have only just begun reckoning with what it means for its primary business operations to be in the material realm. You cannot vibe-code a data center. In perhaps the clearest sign of the AI industry’s desperation to keep building, Anthropic, the most safety-conscious firm, is spending $1 billion a month to rent a data center from Musk, who just a few months ago deemed Anthropic “evil.” Anthropic CEO Dario Amodei, for his part, has accused Musk of “disturbing negligence.” Meanwhile, the tech industry is struggling to quell a mounting AI backlash, which has largely taken the form of opposition to data centers. A few days ago, New York became the first state to institute a moratorium on new “hyperscale” data-center construction.

[Read: The AI backlash could get very ugly]

If anything, AI is poised to become even more like drilling an oil well. All the copper, silicon, electricity, and labor that go into building and running a data center are mustered to process and spit out “tokens”—in essence, words—that are the basic unit of AI. The State Department recently announced an international pact to bolster the AI supply chain, called Pax Silica. “If the 20th century ran on oil and steel, the 21st century runs on compute and the minerals that feed it,” Jacob Helberg, the undersecretary for economic affairs, said in the announcement. Financial markets and investors are already beginning to treat tokens as the raw inputs to modern civilization: Various firms are launching such markets for compute, akin to how you can trade oil, steel, and corn on a stock exchange.

By dominating the digital world, Silicon Valley has already exerted indirect control over the physical one. Today’s data-center build-out is a whole new beast. From the most concrete aspects—cooling, power, land—to the most abstract financial instruments, AI has morphed into a heavy industry.

Updated at 3:41 p.m. ET on July 16, 2026

There are two competing schools of thought over just how water-intensive AI is. In one, the technology is horribly thirsty. Data centers will exacerbate droughts throughout the country and “drain the great lakes,” as one popular Reddit thread puts it. The former Representative Marjorie Taylor Greene has taken up the issue, sarcastically posting at one point: “How dare the peasants complain about data centers stealing their water and driving the cost of electricity!!”

The other school, populated by Silicon Valley types, holds that “The Data Center Water Crisis Isn’t Real,” as an article in the right-wing publication Pirate Wires argued in December. The issue with water “is totally fake,” “insane,” and has “no connection to reality,” OpenAI CEO Sam Altman said a few months ago.

As with so many debates around AI, the arguments are not always presented in good faith. But unlike, say, the ambiguities over how much work a chatbot can or should be trusted to do, water use seems like it should be measurable, with clear facts supporting one side or the other. That is not the case. Does AI waste a ton of water? Not necessarily. Sometimes. It really depends. “We don’t have the real ground-truth numbers on data-center water use to put everyone on the same page,” Eric Masanet, a sustainability researcher at UC Santa Barbara who studies data centers, told me. Neither data-center operators nor utilities provide much insight; what public information exists can be contextualized, recontextualized, and distorted to suggest basically anything.

[Read: Generative AI is an engineering disaster]

Yet the water issue is tremendously consequential—not only for the well-being of residents in all of the counties where new data centers are popping up but also for the future development and regulation of AI itself. Just this week, when New York Governor Kathy Hochul signed the nation’s first statewide data-center moratorium, her office cited the “massive amounts” of water that hyper-scale data centers could require. In a drought-stricken county or during a heat wave, a data center may seriously burden local water supplies. Where water is abundant and utilities can afford new infrastructure, data centers may pose little issue.

Outside of water-stressed regions, “it’s not really a major concern for now,” Fengqi You, an energy-systems expert at Cornell, told me. “But as the sector grows, who knows?” Turning any particular case into a sweeping pronouncement produces little but heightened emotions and confusion. As ever, the truth is far more nuanced and conditional than activists or tech executives admit.

The problems start with the framing. How much water do data centers use? is not a particularly useful question. On a national scale, these buildings do not use an alarming amount of water. In 2023, data centers used slightly more than 17 billion gallons of water for cooling, as much as about half a million Americans collectively use in a year. This may seem like a lot out of context, but it’s less than a tenth of a percent of the total amount of water used by U.S. farms that year.

Things can look different at the county level. A Meta data-center campus under construction in Lebanon, Indiana, could, at its peak, demand 8 million gallons of water a day: This would be a drop in the bucket in New York City, but it’s more than double the peak demand of the town. Every data center demands a distinct evaluation, and the picture can quickly shift. Water use depends on the local climate, the local water supply, and the makeup of the regional electrical grid, as well as on the design of the actual data center. For Meta’s Indiana data center, city officials have said that planned upgrades mean there is “nothing to indicate” that they will run out of water. Meanwhile, in Newton County, Georgia, a local official has said that “we just don’t have the water” for all of the data centers trying to move in and that the county must “race” to improve local water-recycling facilities.

Rather than worrying about the quantity of water, it’s best to ask how a particular data center uses water. In general, water is always used for the same thing—to keep the computer chips inside these buildings cool—but there are many different methods for doing so. Broadly speaking, data centers can either expel the heat those chips generate with cooling towers, which evaporate water, or with air-cooled chillers or similar technologies that use electricity to vent away hot air. Think of cooling towers as human sweat, which passively cools us down as it evaporates, and chillers as car radiators, which use a chemical coolant and then blow air to physically push out the heat.

The AI industry has settled on the car-radiator approach for many of its biggest, most controversial data centers. They run water directly into the servers to pull away heat, cycle the water down through an air chiller, and then—crucially—reuse it. That’s how OpenAI’s Stargate data center in Texas, Meta’s Hyperion data center in Louisiana, Microsoft’s Fairwater data centers across the country, and many other enormous facilities operate for most of the year. Such closed-loop systems don’t lose water to evaporation, which is likely what Altman was referencing when he called concerns over water “insane”: Once these data centers have water, they don’t need more of it, in theory, because it’s never used up.

The real incentive for closed-loop systems is convenience. Building out local water infrastructure or tapping into a reservoir can take years, using groundwater is controversial (groundwater doesn’t replenish quickly, and using it can pollute the environment), and you can’t realistically bring water from Lake Michigan to your data center in dusty Arizona. Water availability can be a serious bottleneck to new data-center construction. A closed-loop system with air chillers bypasses these problems.

[Read: AI is taking water from the desert]

So why doesn’t every data center just use chillers? For starters, air chillers require substantially more electricity compared with cooling towers to expel the same amount of heat. “Can data centers use zero water? Yes, that’s very easy,” Shaolei Ren, an AI and sustainability researcher at UC Riverside, told me. “But the downside, the cost they have to pay,” is using more electricity—10 to 65 percent more, by various estimates. One nine-gigawatt facility being considered in Utah would require nearly as much power as New York City.

Data centers’ almost surreal power demands are largely being met by combustion turbines, many of which are built by the tech companies themselves (to avoid waiting on upgrades to the local grid). And generating electricity—including from these private power plants, which burn fossil fuels—needs water too. In 2024, Meta’s “indirect” water consumption—referring to the water used during power generation—was 19 billion gallons. That’s 23 times its direct water consumption, and most of it went to data centers.

[From the April 2026 issue: Inside the dirty, dystopian world of AI data centers]

Few industries consistently report indirect water use, in part because many of the estimates are flawed. Indirect use comes from averaging the water footprint of all the power sources on a regional grid—which includes hydroelectric plants that consume huge amounts of water due to evaporation from dams and lakes. In other words, indirect water-consumption figures can arguably under- or overestimate any individual plant’s contributions. Paradoxically, the AI industry’s decision to avoid using water to dissipate heat from inside its data centers might expand some facilities’ overall water footprint. “If you use water to make your cooling more efficient on-site, you will use less electricity,” Jonathan Koomey, a data-center and sustainability researcher, told me. “It’s not a simple matter of water use bad on-site.”

It might be possible to design data centers that use both less electricity and less water. In cooler climates, for instance, data centers can pump in outdoor air; this is what Amazon does at its mega data center in northern Indiana. In summer, spraying small amounts of water into the outside air can bring down the temperature enough to do the same. Data centers are also beginning to experiment with allowing the chips to run at higher temperatures, which reduces the overall cooling demands. And, of course, the rather obvious way to build data centers that put less strain on the environment is to opt for renewables—which consume almost no water—instead of natural-gas turbines.

Until these approaches are the norm, people will understandably worry about the effects these sprawling facilities will have on their communities. Assessing how any data centers will affect nearby reservoirs ultimately requires the scarcest resource of all when it comes to these facilities: accurate and precise information. Tech companies want data centers up and running quickly, so they contract with shell companies that negotiate behind closed doors and under nondisclosure agreements. Residents might find out about new construction only after shovels are in the ground.

Already, rising temperatures and prolonged droughts are out of any individual’s control; now a big industrial facility arrives, under the cover of night, that may or may not exacerbate many problems caused by the climate crisis. This sense of confusion goes a long way toward explaining why data centers’ environmental impacts have become one of the key battlegrounds in the AI backlash—the primary issue, in many cases, is not lack of water so much as lack of control.

This article originally referred to a facility under construction in Utah. In fact, that proposal is still being considered, and construction has not begun.

Sign up for our newsletter about national security here.

In recent weeks, Silicon Valley has been fawning over an AI model released by a lab in China. The program, called GLM-5.2, has been called a “marvel,” “very good,” and a “step change.” The billionaire venture capitalist Marc Andreessen posted on X that “AI insiders are saying GLM-5.2 is the first Chinese AI model to match and often beat” the top public U.S. models. Guillermo Rauch, the CEO of the AI-coding platform Vercel, said that he is “genuinely impressed, almost shocked” by GLM-5.2’s coding abilities. Or as one AI founder at a San Francisco dinner party recently told me: “Praise GLM-5.2.”

In a sense, GLM-5.2 is China’s answer to Claude Code, Anthropic’s agentic assistant that has reshaped the AI boom. This has been the year of AI agents—tools that don’t just chat but promise to do stuff on your behalf, whether coding a website or booking a vacation. Chinese AI models have been steadily improving, but none previously proved capable or consistent enough to be used as agents. Now GLM-5.2, developed by the Chinese company Z.ai, rivals some of OpenAI’s and Anthropic’s top offerings and, by many measures, has leaped ahead of Google Gemini. And it is several times cheaper.

[Read: Move over, ChatGPT]

Despite Silicon Valley’s awe over GLM-5.2, an inexpensive competitor couldn’t have come at a worst time for America’s frontier AI labs. Having successfully persuaded corporate America to give their products a try, OpenAI, Anthropic, and Google are now struggling to prove that their tools are worth the money. These bots can be very expensive to use, running up bills into the thousands of dollars, per employee, per month. Uber reportedly spent its entire 2026 budget for Anthropic models in only a few months. Other Big Tech companies including Meta, Amazon, Tesla, and Adobe are also reportedly clamping down on employee AI usage. Citi at one point shut down employee access to OpenAI’s and Anthropic’s most expensive models, according to reporting from 404 Media (which Citi has contested).

Even though political leaders and tech executives have framed the U.S. as in the middle of a contentious technological race with China, Americans can still use Chinese models. While it’s too soon to know whether GLM-5.2 is really capable of replacing America’s top-tier AI agents, any firm or developer who is balking at the costs now might have an alternative. The arrival of GLM-5.2 poses a business dilemma for Silicon Valley—and possibly a national-security dilemma for the country as a whole.

In some ways, America’s AI industry has been here before. Many previous impressive Chinese AI models have not triggered a mass exodus of OpenAI and Anthropic customers, with one major exception: In January 2025, after DeepSeek launched a cheap AI model on par with America’s best, adoption of Chinese AI models leapt up. Within two months, the share of global web traffic to Chinese AI models jumped from roughly 3 to 13 percent, according to research from RAND. OpenAI, Anthropic, and Google quickly responded to DeepSeek with cheaper models of their own, and then the rise of AI agents early this year seemed to solidly put American labs back on top.

[Read: China’s DeepSeek surprise]

Even before Z.ai launched its new model, there have been quiet signs of a shift to cheaper, Chinese models amid mounting concerns about AI bills. Over the first five months of 2026, DeepSeek’s adoption among the 70,000 U.S. firms that use Ramp, a financial-operations platform, increased from 0.1 to 0.3 percent, Ara Kharazian, Ramp’s lead economist, told me. Kharazian added that 6 percent of Ramp customers that spend money on AI use third-party platforms that provide access to many different AI products. On OpenRouter, one such platform, the six most popular AI models are Chinese; in under a month, GLM-5.2 already ranks fifth. These data don’t capture all the software developers and companies directly downloading Chinese AI models, which are typically open source, and configuring them on their own computers—that is, largely scrappy start-ups and academics who don’t have the budget to use a fintech service like Ramp. Chinese models accounted for nearly half of all open-source AI downloads from February 2025 to 2026, according to data from the popular AI platform Hugging Face.

Meanwhile, OpenAI and Anthropic keep releasing more, rather than less, expensive products. And soon, they will likely have even more competition. The Claude Code and broader AI agent frenzy started seven months ago, which is also approximately how far behind Chinese AI firms have been lagging in model development. China’s other labs, including DeepSeek and Moonshot AI, will almost certainly release similarly capable and inexpensive AI agents before long. Coinbase, a popular crypto company, claims to have nearly cut its AI spending in half by defaulting to cheaper models including GLM-5.2 and Kimi, another popular Chinese bot. “The scenario to worry about is China has good-enough models at a quarter of the price,” Kyle Siler-Evans, an AI researcher at RAND, told me. “I think that is likely the future we’re headed toward.”

Kharazian cautioned against overstating this narrative. Whereas some tech-forward firms are pulling back on their AI spending, most American companies already spend very little on the technology: The median Ramp customer spends just $11 per employee on AI. “Not to say that the rest of the market won’t go in that direction,” Kharazian said, but Anthropic and OpenAI will have “ample time to respond” with competitive pricing—as they did with DeepSeek.

Perhaps the biggest obstacle to the ascendance of GLM-5.2, at least in the U.S., will have little to do with model capabilities or pricing. There are serious concerns about Chinese firms essentially using their AI to hoover up sensitive data and steal corporate secrets. The perceived risks of using Chinese technology could create a profound chilling effect among the customers of any U.S. firm—not to mention the uncertainty created by the possibility of federal regulation. Consider that Chinese electric vehicles are by all accounts better and cheaper than their Western counterparts, but they can’t be bought in the U.S.; one possibility is a near future in which Americans are functionally or legally barred from accessing the most cost-effective AI models.

The biggest implications of GLM-5.2 and the coming onslaught of cheap Chinese AI agents may also be geopolitical rather than economic. Whatever soft power the U.S. gains from its technology may wither as software developers in countries with less tense relationships with China turn to DeepSeek and GLM-5.2 in greater numbers (consider how the EU, U.K., and Canada are importing Chinese EVs). Meanwhile, despite Big Tech spending ever greater sums on advanced AI chips to train their models—chips that are banned from export to China—the gap between U.S. and Chinese models has not widened; if anything, it may be shrinking. That means any military, economic, cyberoffensive, or other advantage that AI can grant the nation could be vanishing. America’s lead in the AI race, for the first time since DeepSeek, is at real risk of slipping.

On a Friday in April, I hopped into an Uber to a fish market in San Francisco with a couple of tech founders on a mission to buy lobsters. Not for dinner, but for science: The duo dreamed of one day altering human consciousness, but they would start by toying around with some crustaceans. They intended to perform neurosurgery on the lobsters in the hopes of controlling them with an AI bot.

Leading the way was Elliot Roth, a bearded 32-year-old wearing a black T-shirt with Longevity printed across the chest and a silver chain with a double-helix pendant. To push the boundaries of the five senses, Roth has implanted a magnet in his left ring finger. He told me his nerves have grown around the magnet, giving him some sort of magnetoperception—he can feel when a microwave turns on in another room and sense when a radio tower is nearby. In the car, Roth took off his watch and allowed it to dangle, magnetized, from his finger.

Roth did not have any experience with lobster surgery, so he had enlisted a co-conspirator: William Joy, a lanky, 19-year-old redhead who also had never operated on a lobster but seemed confident in his fine motor skills. Joy would modify an off-the-shelf kit that can be used to remote-control a cockroach and implant the controlling device in the lobsters. If successful, Roth and Joy would then be able to send targeted electrical signals to direct the lobsters’ movements and, hopefully, the pinching of their claws. The final step: connecting the lobsters to the popular AI agent OpenClaw, which uses a lobster for its logo (get it?), and allowing the bot itself to decide what the lobsters should do. Perhaps the lobsters could even be made to control OpenClaw, Joy excitedly told me. “I’m pretty sure it’s going to be the first real instance of a complex AI agent interfacing with a biological organism,” he said during the ride.

With two lobsters secured in a pink plastic bag, we returned to an office building in downtown San Francisco where Roth and Joy would conduct the experiment. They began setting up an aquarium tank, fiddling with the water’s salinity and acidity. The duo claimed to be very concerned about animal welfare. “We are going to give a lot of thought to, How can we ensure that they don’t suffer?” Joy said, adding that he would look into giving them some kind of anaesthesia. These particular animals were “already destined for the dinner table,” Roth said, and they planned to later eat the lobsters out of respect.

This experiment was eccentric but not outlandish—at least not to Roth and Joy’s housemates. They both are residents of Biopunk House, a repurposed college dormitory inhabited by more than 20 aspiring entrepreneurs with dreams of curing aging and rewiring the human brain. In the lingo of San Francisco, this kind of co-living situation is called a “hacker house.” Out of sheer convenience, if nothing else, young tech founders have long decided to band together to make rent in one of the most expensive parts of the country. Mark Zuckerberg and some other early Facebook executives famously lived together in the 2000s.

In recent years, hacker houses have become more deeply woven into the Bay Area’s culture than ever before. During the AI boom, crashing in a group house is something like a rite of passage for any young AI founder. Across San Francisco, hacker houses are filling up with college dropouts and, in at least one case, exuberant tinkerers spending their Friday buying lobsters for science. They want to change the world and become spectacularly wealthy, but first they’ll have to share a bathroom.

A couple of days before the lobster mission, I rolled my suitcase up to the front door of a slightly battered Victorian row house a few blocks from San Francisco’s famous Painted Ladies. This was the Accelr8 house, where I would spend the next week among half-a-dozen entrepreneurs. What better way to understand the hacker-house scene than to live in it?

Accelr8 has a more modest setup than the Biopunk House. There’s no theme other than trying to launch a tech company with little to no funding. After I’d settled into my room, which included a wobbly standing desk and a view of the adjacent building’s wall, I got a brief tour of the house. It has two floors, each with three bedrooms, two bathrooms, and a kitchen. On the top level is a shared workspace with desks and a small bookshelf featuring Atlas Shrugged; The Art of Power, by Nancy Pelosi; some Foucault; and three vintage issues of Playboy. Below, a living room is crowded with couches, succulents, and kitschy art. Nearly every room in the house had an Amazon Basics odor eliminator (lavender), and red placards implored residents not to leave dishes in the sink, although dirty dishes were consistently there. That first day, as I put some yogurt away in a communal fridge, a faint sour smell wafted out. All things considered, this was one of the cleaner hacker houses I’d been to.

I soon began to meet the other residents. Michael Adams, a former rocket engineer with long, flowy hair, barged in while I was sitting in the workspace. “We’re going to the moon!” he said, and showed me a livestream of Artemis II hurtling toward space. During the pandemic, Adams had become concerned about and then obsessed with the failures of San Francisco’s local government. He quit his job at a space start-up and eventually built CivLab, a platform that, among other things, offers an interactive map that overlays local news, police reports, and community events across the city. Humans can comment on the site, and to further provide the atmosphere of a public forum, Adams had created a group of AI agents (with names such as “Max Entropy” and “Pearl Plexity”) to post about various happenings.

Adams and most of my other housemates were in or around their 30s. Kai Song Eer and Jay Yen Lim, who had just moved here from another hacker house in San Francisco, were building an AI agent to automate sales calls. On my first day, they offered me some (delightful) roasted oolong brought from Malaysia. Alan VanToai, whose room was on my floor, had an awakening to the potential of AI about a year ago when taking a psychedelic. He had recently vibe coded an AI-guided-meditation app.

And then there was Daniel Morgan, a 30-year-old Texan who lived in the room below mine. Morgan co-founded the Accelr8 hacker house with his friend Patrick Santiago in 2024, and it’s a sort of start-up in its own right. Each room in the house has a rent of about $3,000 a month, which Morgan told me was enough to basically break even on the lease and other expenses. Accelr8 has also launched another location, a “hacker hotel” that offers lower prices and fewer amenities, and it is where Santiago lives.

On my second evening at the house, I lounged in the backyard with Morgan and Santiago, surrounded by lawn furniture and a rusting barbell. It was a typical San Francisco evening: sitting outdoors to sip ginger beer, munch on sour-cream-and-onion Pringles, and talk about how AI is warping the world.

When it got cold, we went inside and settled on some couches in the living room. Morgan offered to cook what he described as “hacker-house slop bowls”—stir-fried noodles with eggs, mushrooms, and greens. While we waited, Santiago projected a YouTube show onto a large screen. It was a miniseries inspired by the residents of Crypto Castle, a hacker house from the 2010s. In one episode, a journalist visits and brings up Elizabeth Holmes, the Theranos founder who, at that point, had just been accused of running a massive scam. She may have made some mistakes, one resident of Crypto Castle said, but she “had a vision.” The hacker-house residents beside me laughed.

Hacker houses are popping up everywhere in San Francisco. “You could just walk a few blocks” in many neighborhoods, Amber Yang, an investor at Lightspeed Venture Partners, told me, and “there’ll probably be a few houses of start-ups.”

People who want to found an AI company have little choice but to head to Silicon Valley. Nearly two-thirds of the world’s AI-start-up investment, some $120 billion last year alone, is made in the Bay Area. The gold rush has made living in San Francisco even more difficult: Rents have risen roughly 15 percent over the past year and are likely to continue soaring as the public offerings of SpaceX, Anthropic, and OpenAI mint scores of AI millionaires. Adams put it bluntly: He and his partner could not afford their own one-bedroom apartment, so they moved into Accelr8. My stay at Accelr8 was itself motivated, in part, by a scramble to find a roof for a weeklong stretch between rentals in San Francisco. (I am not launching an AI company, but I did pay prorated rent.)

In the process, hacker houses themselves are changing. A number are becoming more professionalized and lavish. Down the block from Accelr8 is perhaps the most prestigious of them all, HF0, a mansion obscured by tall hedges that has raised a $100 million fund to invest in its residents. And the Biopunk House—where Roth and Joy, the two DIY lobster surgeons, live—is part of the Residency, a network of houses that counts Sam Altman as an adviser. The Residency collectively has an acceptance rate of 3 percent (and falling) and offers a three-month stay in homes where laundry, meal preparation, cleaning, social events, and basically any adult tasks are taken care of for you. “That’s 20 hours a week” back to dedicate to your company, Nick Linck, who founded the Residency, told me. We sat in the media room of one of his houses, a Noguchi table between us and fake ivy on the shelves behind.

The house where I stayed was hardly luxurious, but it was still a major step up from Accelr8’s original location, which had 15 bedrooms and a single kitchen for $1,500 a month. When I first met Morgan there last spring, I spotted a sign over the sink admonishing residents to Wash your pans or Sam Altman will get you. (Some things, I suppose, haven’t changed all that much; Adams would later dub me the “dish fairy,” on account of my putting away the dishes every morning on our floor.) Accelr8 just opened a second hacker hotel, which it markets as an all-inclusive residency—daily breakfast, full access to hotel amenities—and is trying to raise some $20 million to purchase another property to convert into a hacker house.

Too many companies, too many products, and too many venture capitalists are coursing through San Francisco. Hacker houses can help investors screen for genuine talent. Some houses and their investors take equity in residents’ start-ups. So-called angels—wealthy people who use their own money to fund companies—can drop by a hacker house, chat with someone in the living room, and cut them a $100,000 check on the spot. Still more venture capitalists visit for product demos.

For all of the pragmatic benefits of a hacker house, there is also a more soulful draw. Starting a company is lonely, time consuming, and risky. I met hacker-house residents who were spending down their life savings or had asked friends and family for loans. Most of them will fail. At the Accelr8 house, between sprints of coding and sales calls, the founders sat on couches and chatted about good date spots, or wandered to the bar next door to watch basketball. They were friends as much as they were housemates.

On my fourth day at Accelr8, Morgan barged into the kitchen carrying an armful of mugs that had been left out the night before. He said he sometimes felt like a “hacker-house dad.” After pouring himself coffee, Morgan took a gallon of 2 percent milk out of the fridge, smelled it, made a disgusted face, and put the jug right back where he found it. That afternoon, I ran into VanToai and Adams laughing in the kitchen. Adams was preparing coffee and had poured the remnants of a jug of 2 percent milk—that milk—into his glass without first smelling its contents. The milk, they joked, was “imbued” with the curdled whey. VanToai and I watched with incredulity as Adam tried to strain out the curdled bits. He gave up and reached for the instant coffee.

Every Sunday, the Accelr8 community members—both current and former residents—gather at the main house to demo their products at dinner. During my stay, it was hot-pot night. A couple dozen founders stood around two pots of simmering broth and ate thinly sliced beef, enoki mushrooms, and other fixings from paper bowls. With some, I discussed superintelligence and the coming automation of labor; with others, how superintelligence was mostly hype, AI is addictive, and sometimes agents “just make you feel like you’re getting somewhere” without accomplishing much. But everyone was enthusiastic to pitch their start-up.

Tigran Voskanyan, who was staying in Accelr8’s hotel, is developing an AI assistant that, he told me, centralizes your media, allowing you to access Spotify, Telegram, and so on in one place. Voskanyan asked me to pull out my iPhone and load his website so he could show me the program. This proved much slower than just opening my music app, and the bot had some voice-recognition difficulties. As I put my phone away, Voskanyan explained that his prototype is in its early stages, and he’s still working out some technical issues. Suddenly, he was interrupted by a robotic voice emanating from my jeans pocket. Voskanyan’s AI, despite being closed and my phone being asleep, was listening. I hurriedly revoked the bot’s microphone access.

Other products I heard about that evening included the crypto-based payments system Fumav, for “Fuck you, Mastercard, American Express, and Visa,” and a tool that was described to me, without irony, as “a new format” for the book. This turned out to be a text-based video-game app played on an e-reader, seemingly more akin to reinventing Oregon Trail. Matt Thompson, the founder who had made this tool, had hand-traced every illustration in his sample game to give it an artisan aesthetic. While we were cleaning up after dinner, Morgan told me that if he had the money, he would invest in Thompson’s start-up immediately.

The earnestness within these hacker houses is at times endearing and frequently confounding. Here is an ecosystem of people who generally fear the path that AI could lead us down and yet are eager to sell AI products, who see the first step in making the world better as conquering it with their technology, who earnestly believe that avarice is a prerequisite to altruism. In this city, for an idea to matter, it must exist as a product. To improve the world, you must turn people into users. Scale is a means to an end, until it isn’t. As Sourya Kakarla, a former Accelr8 resident I met at demo day, put it to me, he was “initially attracted to start-ups for the heart,” and a few years ago built an app to help elders in rural India use smartphones. But right now, he said, he “has to navigate short-term market realities to keep the heart alive.” He’s doing AI-consulting gigs to make money, like so many founders I met.

These tensions are baked into the hacker house itself, where young entrepreneurs scrap along together until they make enough money to leave the pack behind. Morgan was well aware of this dynamic, and expressed complex feelings about technology. One evening, walking back from dinner, Morgan lamented how in San Francisco, many people will stop mid-conversation and pull out their phone to request AI input. “How about what you, a human, have to say?” Morgan said, incensed. He would later tell me he’s worried about the “ChatGPT babies” growing up dependent on AI. But at the same time, he is spinning up his own AI-consulting firm.

On my last full day at Accelr8, I watched Morgan take a call with executives at a boutique construction company. Seated at the living-room table and wearing a dress shirt tucked into sweatpants that were outside his camera’s view, Morgan explained how he could build them a team of bespoke AI agents. “Sales people definitely need this,” one of the executives told him.

By then, Accelr8 seemed to be crumbling around me. After hot-pot night, the sink clogged and the upstairs dishwashers broke (the one downstairs wasn’t working when I arrived). I was holding my breath while opening the fridge. The morning I was set to leave, much of the crew gathered in the living room to see me off. I had graduated from Accelr8 and was added to an alumni group chat. Morgan handed me a souvenir, a glittering Accelr8 sticker for me to put anywhere. “Preferably a laptop,” he said.

I did still have some unfinished business. A couple of weeks after the trip to the fish market, I checked in with Joy to see how the lobster surgery had gone. We sat down in the same room where he and Roth had set up their experiment, but the tank beside us was conspicuously empty of any life or, for that matter, liquid. The lobsters had died before surgery. Perhaps they’d gotten the salinity of the water wrong, Joy told me. He had conducted preliminary dissections on their carcasses, and he said he was going through an ethical crisis about “whether I should even be doing this.” Nobody ate the lobsters.