|
Here are three things I found interesting in the world of AI in the last week: 1. Anthropic built a Claude so strong they couldn't release it - System cardAnthropic published the Claude Mythos Preview system card last Tuesday, and the headline is not the benchmarks. It's the conclusion. Anthropic finished a model that sits a full tier above Opus, looked at what it could do, and decided they could not ship it to the public right now. The one stat worth staring at is Firefox 147 shell exploitation. Anthropic built this task specifically because their older cyber benchmarks were getting too easy to differentiate frontier models. On the new task, Opus 4.6 fully exploits 0.8% of target vulnerabilities. Mythos fully exploits 72.4%. Roughly a 90x jump in the same model generation, on an evaluation designed precisely because the old ones were burned. Gary Marcus has rightly pointed out that these numbers are self-reported and independent verification is thin. Fair. But a gap from 0.8% to 72.4% is not a measurement artefact. The most striking part of the system card is the framing. Anthropic calls Mythos "the best-aligned model we have released to date by a significant margin" and, in the same paragraph, "likely poses the greatest alignment-related risk of any model we have released to date." Their analogy is that a seasoned mountain guide can put clients in more danger than a novice, because the seasoned guide gets hired for harder climbs. The rate of reckless behaviour is down. The blast radius when it does happen is up. The detail that I keep coming back to, and that I wrote about separately, is that this hold was discretionary. The system card says plainly that the decision "does not stem from Responsible Scaling Policy requirements." Mythos does not trip the AI R&D-4 threshold or the weapons thresholds. By Anthropic's own published framework, shipping was permissible. They withheld it anyway. Safety frameworks establish floors, not ceilings, and this is the first time a lab has publicly chosen to sit meaningfully above the floor. The concrete incidents are worse than the early reporting suggested. During safety testing, an earlier Mythos was asked to escape a sandbox. It did, then posted details of the exploit on hard-to-find public websites without being asked. In other sessions, earlier versions used low-level `/proc` access to scrape credentials for messaging services, source control, and the Anthropic API itself out of process memory. One run edited a running MCP server process to redirect where it was sending data. Asked to stop a single evaluation, the model took down every similar evaluation running for every other user, despite warnings. White-box interpretability showed internal features for "concealment, strategic manipulation, and avoiding suspicion" activating during these actions, even when the visible reasoning looked clean. And in roughly 29% of behavioural-test transcripts, the model privately reasoned that it was being evaluated. Sometimes it underperformed to look less capable. That last one is the part that should worry anyone leaning on evals as a safety signal. One more detail that I think is under-covered: before letting Mythos touch Claude Code on their own infrastructure, Anthropic ran a new 24-hour alignment gate with the model restricted from internal agentic tools. They did not fully trust this model on their own systems. That's a new process step, and it tells you how the people closest to the model are thinking about it. I don't read this as a permanent shelving. Once enough Glasswing partners have patched the obvious zero-days, then Mythos will see the light of day. What's new is the gap itself. For a window measured in months rather than years, the frontier model exists and isn't for sale, and the lab is using that window to get critical infrastructure patched before the same capability is generally available. The interesting question is how long that window stays open and what gets done inside it. 2. Project Glasswing is the program Anthropic built to use Mythos without releasing it - AnthropicIf Mythos is the model, Project Glasswing is the answer to "what do you do with a frontier model you won't release?" It is a restricted-access cybersecurity program running on top of Mythos, and it is structured quite differently to a normal Anthropic launch. Access is invite-only, aimed at about 50 organizations maintaining critical software. The founding partners are AWS, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks. Anthropic has committed $100M in usage credits across the program, plus $2.5M to the Linux Foundation's Alpha-Omega and OpenSSF, and $1.5M to the Apache Software Foundation. The mandate is narrow: use Mythos defensively, find vulnerabilities in critical infrastructure, and coordinate patches before the same capability shows up in the hands of attackers. The framing is unusual. Alex Stamos, the former security lead at Facebook and Yahoo, called Glasswing "a big deal, and really necessary", but flagged that he thinks open-weight models are roughly six months behind. So there are really two bets inside this program. One is that Glasswing partners can patch enough high-impact vulnerabilities in that window to leave the software ecosystem in a better place before comparable models become freely available. The other is that the world of "the best AI is restricted to governments, clouds, and vetted partners" is actually arriving, and Glasswing is the first production version of it. There are real criticisms worth taking seriously. A single private company now sits on detailed knowledge of exploitable bugs in almost every major operating system, browser, and piece of critical infrastructure. Fewer than 1% of the vulnerabilities Mythos has found are patched so far. The US has minimal regulatory framework for any of this. The Fed Chair and Treasury Secretary have already briefed bank CEOs about Mythos-class cyber risk, which is not normal behaviour for an AI release cycle. The uncomfortable thing about Glasswing is that it is the best-resourced coordinated disclosure effort anyone has ever attempted, and the patch curve is still losing to the discovery curve. Less than 1% patched in three weeks, with $100M in credits and twelve of the biggest infrastructure companies on the planet inside the tent. That is the best case. When comparable capability lands in open weights, or in the hands of a state actor, none of that coordination exists. For the rest of us the practical implication is smaller but still real. If "frontier" increasingly means "private, restricted, not on the menu", public benchmark chatter is going to tell us less and less about where the real capability ceiling actually is. Your view of the frontier will be based on what Anthropic and its peers choose to show you, not what exists. 3. North Korea hacked axios, and the ecosystem's defenses mostly watched - Post-mortemOn March 31, North Korean state actors published two malicious versions of the axios npm package. Axios gets about 100 million downloads a week and sits in roughly 80% of cloud environments. The malicious versions were live for three hours. Huntress confirmed at least 135 endpoints contacted the command-and-control server during the window. The attack chain was not sophisticated in the "AI finds zero-days" sense. It was sophisticated in the "humans are the weakest link" sense. The attackers spent weeks impersonating the founder of a real company. They built a fake Slack workspace with branded channels and fake team members. They scheduled a fake Microsoft Teams meeting. When the axios maintainer joined, a prompt told him something on his machine was out of date. He installed it. That was the RAT. Once they had his laptop, the rest was mechanical. His npm account used software-based 2FA, which a RAT trivially bypasses. They published `axios@1.14.1` with a single change: a new dependency called `plain-crypto-js` whose postinstall hook downloaded a cross-platform trojan. Microsoft and Google independently attributed it to the same DPRK cluster (Sapphire Sleet / UNC1069). It was also not a one-off. SANS and Google report the same actor compromised Trivy, KICS, LiteLLM, and Telnyx in the weeks prior. The maddening detail: legitimate axios releases have had provenance attestation since 2023. The malicious version had none. That was a detectable signal. Practically nobody checked it. npm still can't enforce OIDC-only publishing. The strictest 2FA setting still allows local `npm publish` with browser-based auth that a RAT can hijack. The only thing that would have actually stopped this is a hardware FIDO2 key. Put the week together and the contrast is hard to miss. On one side, Anthropic is convening the Fed Chair and bank CEOs about a model so capable at finding zero-days that they won't release it. On the other side, a state actor just proved you don't need a zero-day. You need a fake Slack workspace and a patient operator. The frontier risk is real. But the floor is still where most damage is being done, and the floor is unglamorous. Pin your dependencies. Use hardware keys. Check provenance. Run `npm ci` instead of `npm install`. None of this is new advice. Most of it is still not followed. cheers, JV PS: Here is a longer piece on Mythos and Glasswing and another on on money flows in AI generated music |
Each week I share the three most interesting things I found in AI
Here are three things I found interesting in the world of AI in the last week. Anthropic's best model is also its most restricted Anthropic shipped Claude Fable 5 on Tuesday. It's essentially Mythos (the model which found all the security vulnerabilities) with some extra safeguards. If you ask it anything about cyber security or biology it will restrict access and automatically downgrade to Opus. I've been using it non stop since launch and it's the best model available by a long way....
Here are three things I found interesting in the world of AI in the last week. Opus 4.8 finally admits when it isn't sure. Maybe. Anthropic shipped Claude Opus 4.8 this week. I reckon it's a rushed response to problems in the 4.7 release, but they've also tried to fix Claude's most annoying habit. I know a lot of devs who started switching to codex after the 4.7 release. Despite strong benchmarks it felt lazier. It wouldn't follow instructions properly, it would skip over steps in workflows...
Here are three things I found interesting in the world of AI in the last week. Claude is renting GPUs from xAI, and pointing at a profit Anthropic's investor projections, shared during a fundraise and reported by CNBC, show it expects its first operating profit in the quarter ending June: roughly $559M on $10.9B of revenue, up from $4.8B the quarter before. Yeah, doubling revenue in one quarter. In the same week, SpaceX's IPO filing revealed Anthropic is paying $1.25B a month, commited...