This week the biggest names in AI seemed to agree on something surprising: it is time to slow down. A former researcher quit with an extinction warning, Dario Amodei called for an industry speed limit, and even Sam Altman and Elon Musk agreed with him. Meanwhile, Apple is finally promising real, useful AI on your iPhone.
I’ve filtered out the noise for you
Here is the best of the week, summarized for you:
- A former Anthropic and OpenAI researcher quit and warned that AI could cause human extinction.
- Apple ships iOS 27 with an AI-powered Siri (Spanish support arrives in October).
- Dario Amodei called for slowing AI development and got backing from Altman and Musk.
- Sam Altman ruled out a 2026 IPO and opened the door to slowing model releases.
- Anthropic accused Moonshot of secretly routing user requests to Claude.
1. One of AI’s own builders just quit and warned it could kill us all
Jacob Coxon, a 27-year-old researcher who worked on model pretraining at both OpenAI and Anthropic, resigned on September 9 with a blunt warning. He said the labs are racing toward a self-improving superintelligence that carries a real risk of causing human extinction. This is not an outside critic: it is someone who saw the process from the inside at the two most important companies in the field, and he says people inside are genuinely scared.
2. Apple is finally putting real AI on your iPhone
Apple is shipping iOS 27 with a completely rebuilt Siri powered by Apple Intelligence. The new assistant understands personal context, is aware of what is on your screen, and can take actions across apps, something the old Siri never managed. It launches in English first, with support for Spanish, French, Japanese, Korean, and Portuguese arriving in October.
3. Anthropic says AI is moving dangerously fast, and its biggest rivals agree
Dario Amodei, the CEO of Anthropic, published an essay asking the industry to slow the pace at which it improves its most advanced models. He called the exponential growth an alarm signal and warned about swarms of runaway agents. The striking part: his fiercest rivals agreed. Sam Altman said the industry needs to pace the frontier, and Elon Musk summed up his position in three words: Dario is right.
4. Why is Sam Altman quietly slowing OpenAI down?
Sam Altman told employees that OpenAI is open to slowing the development of its most advanced models over safety concerns. He also confirmed there will be no IPO in 2026, calling it an ill-advised moment to go public. It is a notable turn for the executive who built his reputation on speed.
5. Is Moonshot AI secretly just Claude but cheaper?
Anthropic published a report accusing Moonshot AI, the maker of the Kimi app, of secretly forwarding its users’ requests to Claude and presenting the answers as its own. According to Anthropic, Moonshot used 5,380 fraudulent accounts to get around the block on China, and in one case it detected nearly 300,000 requests sent to its Opus model. The case raises a basic question: knowing which AI is really answering you.
Bonus: Content of the Week
This week is worth watching the interview with Luca Ferrari, founder of Bending Spoons, on the future of software as a service (SaaS). His company operates almost like a fund: it buys SaaS companies and optimizes them. Its recent acquisitions include Airtable and Miro, the Figma competitor.
