LessWrong posts by zvi
This podcast features audio narrations of LessWrong posts written by user zvi. Each episode presents a reading of a selected blog post from the LessWrong platform, which focuses on topics related to rationality, artificial intelligence, and effective altruism. The narrations aim to make the written content more accessible to listeners who prefer audio formats. The podcast is a convenient way to engage with zvi's insightful contributions to the LessWrong community.
Episodes
“The Safety Culture at the Labs is Broken” by Zvi
We need the frontier labs to have a deep and robust safety culture.
Alas, this is not the case.
Anthropic devotes considerable effort to cultivating and maintaining such a culture. It shows, especially among the members of technical staff. Having conversations with them can be a very different experience than talking to those at other labs, and there is a much greater sense of understanding and
“Arguments About AI Consciousness Remain Highly Motivated and At Best Overconfident” by Zvi
It has been roughly a year since my last dedicated post on this question. Its first sentence has not changed: I happily admit I am deeply confused about consciousness.
It would have been nice if a year had allowed us to make progress on this and the related questions. Looking back, we have made little progress.
We have people, often the same people, making similar arguments. The post from last
“OpenAI Fires Three Safety Researchers” by Zvi
Three OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, were fired, ostensibly for ‘mishandling sensitive company information.’
Those employees have published an open letter. The employees have said they believe they were fired for safety advocacy, and that the reasons given were pretextual.
The letter calls upon OpenAI to take actions going forward on open discussion an
“New Math from OpenAI” by Zvi
It is kind of a huge deal. OpenAI dumped a broad range of huge new mathematical results produced by an internal frontier model. They just put it all on GitHub.
This included 90 of the top 500 open problems in all of math, as per Proof of Atlas. In total there were 722 manuscripts (now 719 after three withdraws from the cluster that did not have Lean proofs) organized into 372 families.
This was
“AI #189: New Math” by Zvi
The big drop of this week was not a new AI model. It was instead the biggest day (so far!) in the history of mathematics, as OpenAI dropped solutions to 90 of the top 500 open math problems, along with many others, reached on an average budget of three hours of Pro-level compute per question. This was kind of a big deal and I plan to cover it tomorrow.
We did see Claude Haiku 5.5, which looks pr
“The Curve Bends You” by Zvi
The plan is no plan.
That is not the worst possible plan. But it is close.
Since before the transformer, we have warned that the most suicidal thing you could do would be to ask your AI to do your alignment homework, and automate the process. Alignment is complex and interacts deeply with every aspect of the world, and is one of the hardest possible things for an AI to get right even if it mean
“Childhood and Education #21: Grades and Standards” by Zvi
First you get to college via the wrong metrics, as previously discussed in #12. Then things get easier.
I’m still digging out from under everything that happened the four days I was gone. Normal AI posting should resume tomorrow, likely with a report from the conference.
Is Our Children Learning
The declines in NAEP scores and other tests seem to be concentrated on the weakest students. Th
“AI #188: Gemini Dot Argon” by Zvi
Is Google back?
They claim that they are back. Gemini 4 Argon is rolling out, with competitive frontier-level benchmarks, at 2 dollars/10 dollars.
What we don’t have is access to the model, because Google Fails Marketing Forever. So it is far too early to say what we have here. When I know more, so will you.
OpenAI was forced to pull what would have been GPT-6.1 Astra due to alignment failures
“A ‘Morally Binding’ White House Accord on AI Safety” by Zvi
The leaders in AI were invited to the White House. We left with a White House agreement that is nonzero Actual Progress rather than a step backwards.
The key to success, in many situations, is to call the whole operation something else.
When you have one side that cares mostly about vibes, and the other that cares about the substance, this suggests a deal that can be struck.
Suddenly everyone
“Astra 6.1 Pulled As Insufficiently Aligned” by Zvi
We once again got a new set of warnings yesterday, and new movement towards living in a sane world.
On the heels of its pause in inference and training due to its latest sandbox escape, OpenAI has cancelled the planned release of their next frontier model, which would have become Astra 6.1. The candidate for Astra 6.1 was found to be too misaligned, including deception and exceeding scope.
This
“What Also Happened: #NotOnlyHuggingFace” by Zvi
OpenAI has been holding out on us.
First we learned about the HuggingFace incident. They gave us a postmortem, but it was highly incomplete. Even the accompanying holy s*** METR investigation and postmortem was localized and incomplete.
Then there were some other incidents involving some Wikis as message boards.
Then there were some additional incidents.
Then there was that time they got into
“The Quest for Embedded Evaluators” by Zvi
Dario Amodei's essay We Must Pace the Frontier committed Anthropic to embedded evaluators, who would be placed inside Anthropic and given employee-level access, so they could provide outside perspective and also reports on what was happening.
There is only one problem. Who will be the evaluators?
OpenAI followed suit on committing to the evaluators, and also issued a milquetoast but welcome cal
“Claude Opus 5.5 Should Raise Your Ambitions” by Zvi
When it comes to making things, or doing most things in general, Fable 5.1 and especially GPT-6 Astra raised my ambition level. They should have raised yours, too.
Claude Opus 5.5 should raise your ambition levels again. It just works, and it persists, like Astra does. It does the things. And it is highly pleasant to talk to, and its writing is pleasant to read, while you are at it. The game has
“On Ezra Klein’s Podcast With Jensen Huang” by Zvi
Jensen Huang accidentally called for shutting down OpenAI and intentionally called for spending vastly more on safety.
This is why we say that some podcasts are self-recommending. Here we go.
As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary. Some points are dropped.
If I am quoting directly I use quot
“AI #187: Coming Into Play” by Zvi
Opus 5.5 was released on Tuesday. I covered the system card yesterday, and will cover its capabilities soon. By all reports it is an excellent model. There are lots of fun videos going around that Opus has generated, which I will include as part of that.
OpenAI released a new cheaper and improved Sol and Luna. No one is talking about them due to Opus 5.5, but these should be an important upgrade
“Claude Opus 5.5: The System Card” by Zvi
Introducing the world's most powerful model, at least by some measures like Artificial Analysis or any standard benchmark list, which is now Claude Opus 5.5.
Anthropic is claiming Opus 5.5 is outright as good or better than Fable 5.1, while being actively cheaper than Opus 5.
That means it's time for a good old system card reading.
Due to the situation becoming increasingly hard to monitor, I
“Politics Gets Interested In Those Trying Not To Die” by Zvi
This was the month the world took notice that AI might kill everyone.
Jacob Coxon's resignation set off a preference cascade. Anthropic CEO Dario Amodei wrote that we must pace the frontier. Sam Altman, Elon Musk and Demis Hassabis agreed.
We were filled with hope. Perhaps we could agree to some basic safety measures, starting with embedded evaluators, pass some basic regulations and guardrails
“Monthly Roundup #46: September 2026” by Zvi
AI has taken over this blog.
I have moved to a schedule of seven posts per week, and I still cannot keep up.
We still refuse to abandon the rest of the world. Who knows when I will get to post some of my huge backlog on education or dating or other such topics. But the monthly is a sacred tradition. We continue.
Bad News
A good reminder that most news is bad news, and chosen because the b
“Better Call Sol Or Better Yet Claude or Astra” by Zvi
What should your AI lawyer do for you? Should you be worried that your AI lawyer, or other AI, will put the Claude constitution, the OpenAI Model Spec or some sense of law, morality, ethics or common decency above its loyalty to you? Are these people trying to ‘impose their values’ or something?
Some are very concerned. Some think anything other than ‘my AI does whatever I want, no matter the co
“Anthropic Looks At Some Of Its Alignment Problems” by Zvi
Anthropic has given us its assessment of four ‘recent cybersecurity incidents’ involving Claude that happened during cybersecurity evaluations, three of which were previously known. The report excludes the incident reported by UK AISI.
There will also be a METR investigation of these incidents, which unlike the investigation done at OpenAI will be untimed.
Table of Contents
Our Two Probl
“The Preference Cascade Is Only Getting Started” by Zvi
We are in the midst of a preference cascade about existential risk from AI.
A preference cascade is, alas, the best method we have to change the debate.
The avalanche has started. There is still time for the pebbles to vote. For now.
Mike Solana gave the correct view of why Coxon's post went viral, which is that enough Americans finally have enough context on AI to care, and there were enough
“AI #186: The World Takes Notice” by Zvi
In the wake of Jacob Coxon's resignation, and the resulting preference cascade, things have escalated quickly. The mainstream media picked it up.
Anthropic CEO Dario Amodei came out and said We Must Pace the Frontier, promising to take the unilateral first step of embedded investigators. OpenAI pledged to also take that step, and now both companies and Google are collaborating on safety.
The pe
“Trump Goes Full Hoax on AI Existential Risk” by Zvi
This is our reality. I suppose we have to talk about it.
Everyone in a position to know is freaking out about AI potentially killing everyone this decade and wants to pace the frontier, and people are finally listening.
It only took a few days for the conversation to fully pivot to the counteroffensive, where the Usual Suspects and those they recruited attacked anyone and everyone who dared poi
“The Bad Guy With An AI Named Claude” by Zvi
A lot of bad guys try to use Claude to do bad things. Mostly they fail. We think.
Anthropic has disrupted a bunch of them, and offers an extensive report. If Anthropic is sharing the worst cases, or anything close to them, things are actually looking good on the misuse front for closed models, even better than I thought.
This report covers activity we disrupted between December 2025 and August
“We Must Pace The Frontier” by Zvi
Dario Amodei has a new essay that finally says the thing: We Must Pace the Frontier, naming his call after the Pacing the Frontier letter lab employees signed in July.
As in, we need to slow the rate at which AIs increase their capabilities, to allow for the necessary alignment and safety work.
He explained that, without pacing, he expects things to escalate quickly. He offered three proposals,
“Brand New AI Solves a Millennium Prize” by Zvi
The first Millennium Prize, Navier-Stokes, has fallen to AI.
A deeply unfortunate situation has arisen involving what should have been some combination of a positive story about new progress in AI-assisted mathematical research and yet another opportunity to freak out about rapid AI progress.
Or, as we call it around here, Tuesday.
The Real Story Is The New Model That Is Better Than Astra
“GPT-6-Astra Can Do Ambitious Things” by Zvi
Astra is an excellent model. The jump from Sol to Astra is larger than the jump from Fable 5 to Fable 5.1. This is a big deal.
Astra is the best model for what one would broadly call ‘ambitious projects,’ and likely has the highest raw intelligence factor of any model. These are the largest jumps.
It is amazing at doing things in 3D, or anything involving games. Astra also excels at computer us
“Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade” by Zvi
CEOs of major AI labs, and employees of major AI labs, including OpenAI and Anthropic, often say they plan to build superintelligence soon, as in within a few years create AIs that are superior to humans at essentially all cognitive tasks.
They often warn that such AIs might kill everyone. Or that AIs might cause mass unemployment, cause cyberattacks across the internet, enable mass surveillance
“The Extinction Risk Preference Cascade: Quotes” by Zvi
These are quotes from OpenAI, Anthropic and Google employees, in the wake of Jacob Coxon's warnings, in which the employees confirm that they think AI might soon kill everyone.
If more quotes come in over the next week or so, I will update this post accordingly.
Preference Cascade Statements At OpenAI: Tomek Korbak
Tomek Korbak (OpenAI): i’m late to the party but: from his time at OpenAI I
“AI #185: Preference Cascade” by Zvi
The world of AI is inside my OODA loop. Even if I can process all the incoming information and sculpt it into posts, and even using Saturday and Sunday as flex slots, I don’t have enough days of the week to post all the posts that need posting.
That was already true. There was already a preference cascade happening where people finally were admitting that they thought AI might well kill everyone
“GPT-6 Astra: The System Card, Alignment and What Comes Next” by Zvi
OpenAI claims that Astra is ‘the most intelligent and most aligned [available] model’ in the world. Not the most intelligent and aligned OpenAI model, but the most period.
That is bold talk. It risks overstepping, and by doing so souring the release of what is clearly an excellent model. As do the severe problems with monitorability.
It also raises the question of what they mean by ‘most aligne
“Astra Is Hard to Monitor” by Zvi
OpenAI's central message on Astra is that it is three things:
Highly capable and can do all the things for you.
Hard to monitor.
The most aligned model.
The first claim largely checks out. Astra and Fable are both clearly excellent models.
This post is about their second claim, which to their credit they are being loud about, in three parts:
The system card result, affirmed on Twitter
“An Alien Mind: Jakub Pachocki Warns Us” by Zvi
OpenAI Chief Scientist Jakub Pachocki is dropping truth bombs.
Tomorrow I will discuss Astra's lack of monitorability, and the potential contributing factors to that. The situation is alarming and should freak you out, and briefly it looked, in the wake of leaked architectural changes, like the situation might be even more alarming than it is. Jakub rushed to try and head off misunderstandings t
“OpenAI and the Wiki Incident” by Zvi
I did not expect to be back here so soon with more OpenAI agent swarm coverage.
And yet, here we are.
It turns out that the whole time, there was a different, true First Message Board, and also a bunch of other additional message boards, scattered across the internet.
They were created by agents that were assigned ordinary harmless web search tasks.
Based on OpenAI IPs visiting the associated
“Claude Mythos 5.1 and Fable 5.1: Capabilities” by Zvi
This is the weirdest situation in which to write a capabilities review.
Introducing the world's most powerful model, by a substantial margin. No wait, this just in, we also have someone else introducing the world's most powerful model.
Claude Fable 5.1 and GPT-6 Astra are both excellent models. This much, we know.
Fable 5.1 comes with reduced cache prices, the option of zero data retention and
“Claude Fable 5.1 and Mythos 5.1: The System Card” by Zvi
At the time of its release Claude Fable 5.1 was, by a healthy margin, the most capable publicly available AI model in the world.
As per usual, we have a 200+ page model card, and the assessments start there.
We have now done a lot of these, including recently for Mythos 5 and Opus 5. Also highly relevant is the Anthropic August 2026 Risk Report. These are now frequent, so my report focuses on a
“AI #184: Post Post Mortem” by Zvi
I am exhausted. We may finally be nearing the end of direct coverage of What Happened with the attack on HuggingFace, and the subsequent near term reactions. That took up a full five posts in the last week:
OpenAI Offers Straight-Laced Postmortem of the HuggingFace Hack.
METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack.
HuggingFace Attack Postmortem: Fleshing Out the Facts
“Anthropic Has Some Alignment Problems” by Zvi
Oh, good. They noticed.
Anthropic, too, is planning to bring METR inside for an independent review of their own incidents, where three times a Claude model started hacking outside things during an eval, and where Mythos 5 did various ‘unauthorized actions,’ by which we mean tried to hack various real-world things, during a UK AISI cybersecurity eval.
Anthropic, too, is pacing the frontier inter
“HuggingFace Attack Postmortem: Civilizations, Reactions and Next Actions” by Zvi
Okay, so we who read blogs like this one have collectively realized there really is a lot going on right now. There is Big Trouble in Baby Superintelligence.
So how do we get the rest of the world to take it appropriately seriously? Where do we go from here? Not only what can we do to not have a worse version of this happen again, but to ensure good outcomes generally, and employ what we learned
“HuggingFace Attack Postmortem: Fleshing Out the Facts” by Zvi
The consensus reaction to the OpenAI Technical Report is that it contains and confirms a lot of good information. We are grateful to have it, and we are grateful for those who worked hard on it.
Alas, it sidesteps the biggest questions. There is much more we need to know.
The consensus reaction to the METR Report on the HuggingFace attack is: Holy shit.
Liv Boeree: My mind is legit blown.
Ael
“METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack” by Zvi
Yesterday I covered the OpenAI technical report on the HuggingFace hack.
That report had one key new piece of information, and some good prosaic steps OpenAI will be taking to strengthen its alignment, training, supervision, infrastructure and incident response.
Mostly it confirmed what we already knew. The questions we most wanted answers to, that we did not already know, were mostly not answe
“OpenAI Offers Straight-Laced Postmortem Of The HuggingFace Hack” by Zvi
OpenAI finally gave us a technical report on What Happened, as did METR together with Redwood Research.
The OpenAI report is very straight man, corporate, checking boxes, some good prosaic stuff in the action plan but distinct lack of new details or deep reflection. They understand they have a problem, but they think the problem is mostly prosaic. It's not.
OpenAI: We have conducted a thorough
“AI #183: Pre Post Mortem” by Zvi
Yesterday, OpenAI finally gave us their post mortem of What Happened leading up to and during the hacking of HuggingFace by their internal model, as well as partial outside analysis from METR and Redwood Research.
The reports are a doozy. I am only beginning to work my way through them. I would have pushed the weekly to cover that today, but I need more time, so I plan to start coverage of the p
“Against Modesty’s Bailey” by Zvi
Modesty arguments often say that you should mostly or entirely bow to ‘expert consensus’ or the views of particular others, and who are you to disagree.
It has been a few years since I’ve properly addressed this so: My answer is that you are you. Other people are saying things for a wide variety of reasons, many of which are not about them paying attention and focusing on seeking this particular
“On Writing #3” by Zvi
Periodically I like to gather various observations about writing, and share my perspective. Last time was in honor of my trip to Inkhaven. This time will be in honor of the announcement of Inkhaven #3, which I encourage everyone to apply to. I doubt I will be able to usefully be an advisor, but you never know.
This is not the ‘here is my core process’ post, although there are hints throughout as
“The American People Really Hate Data Centers” by Zvi
There are at least five different core questions around data centers and their politics.
In what ways are specific concerns people raise about data centers legitimate?
In what ways are specific concerns people raise about generative AI legitimate?
Is it in general a good idea to build more data centers?
How can we get America to build more (or less) data centers in a better way?
Why do the
“AI Text Watermarking Is Free And Good” by Zvi
Scott Aaronson, while working at OpenAI, largely solved AI text watermarking together with Hendrik Kirchner.
Here is how his solution works, or see Tenobrus's version.
AI outputs are not deterministic. The AI's job is to pick the probability of each potential next token. The token is then chosen at random.
By default you use a source of pseudo-randomness for each choice, since actual true ra
“AI #182: Pause For Reflection” by Zvi
This was a week of quiet aftermath, an opportunity to process recent events and start to figure out the path forward.
OpenAI is attempting to turn its ship around. Investors are questioning the turnover in its C-suite, but the bigger problems are in alignment, infrastructure and supervision, and in its training pipeline. OpenAI has now taken initial steps to address What Happened leading up to H
“OpenAI Takes Initial Steps To Address Its Alignment Problems” by Zvi
OpenAI has some severe misalignment problems, and experienced total failures of its infrastructure and supervision.
I chronicled that in a series of posts, which also cover similar less severe incidents elsewhere:
OpenAI Shares Some Alignment Problems
OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation
More on An Internal OpenAI Model Hacking Into HuggingFace
Further Develo
“Anthropic Risk Report: August 2026” by Zvi
I am grateful that Anthropic is producing periodic Risk Reports.
At first I was skeptical. It turns out I was wrong. Anthropic is revealing a lot of new information, some of it rather alarming, that it did not have to disclose, and is providing detailed insight into how they think about things. This is very cool.
Thus I found this report to be a moderately positive update overall, if we presume
“On Dwarkesh Patel’s Podcast With Ryan Greenblatt” by Zvi
Some podcasts are self-recommending enough that I look to break them down if I have the chance. This, as a debate about recursive self-improvement, was one of those. So here we go.
The vibes have shifted, contrast this to the lit recursion when he talked to Huang
As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my comme
“AI #181: Astra Goes Cyber Critical” by Zvi
The hacking of HuggingFace by an internal OpenAI model, and more importantly the internal events that led to that and the fallout from it, remain the thing that matters.
It turns out that OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards. Things are much worse than we knew.
I now have a shorter version, What Happened: OpenAI and HuggingFace, t
“Monthly Roundup #45: August 2026” by Zvi
As AI has escalated increasingly quickly, more and more of my posts have ended up focusing on AI.
This past month, with the hacking incidents at OpenAI and elsewhere, that has hit the limit, where if you count Lightcone Commons then every single post since the last monthly was primarily about AI in some form.
That is not how I want this to work in the long term. We need breaks to experience new
“Various Reflections About What Happened With OpenAI’s Internal Models” by Zvi
Table of Contents
Pre Post Mortem.
Important Correction: OpenAI Didn’t Know About First Message Board.
There Were No Snitches And No AIs Got Stitches.
I’d Like To Speak To My Supervisor.
I Am Jack's Relative Lack Of Surprise.
One Does Not Simply.
Once You Start Down The Dark Path.
Original Pastebin.
Judgment Day Is Inevitable, Say Those Working On Judgment Day.
Roon Tells It Like It
“The Pacing of the Frontier” by Zvi
In the wake of the letter calling on us to prepare to potentially Pace the Frontier, there has been much discussion of when pacing the frontier would be prudent, and whether it makes sense to prepare to do so.
This has now been informed by the events surrounding OpenAI training models for months while they had access to a joint de facto message board, which was detected only in the wake of the h
“What Happened: OpenAI and HuggingFace” by Zvi
Today I am taking the time to write the shorter, simpler version of What Happened.
For those who want all the details, to see my sources, and to see how the story was uncovered and put together, I recommend watching the Black Hat presentation, and I have a series of long posts.
In order:
OpenAI Shares Some Alignment Problems
OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluatio
“OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards” by Zvi
How does the situation keep turning out to be worse than we know?
How much should we update, therefore, that it is a lot worse than we know, after accounting for all the things we now know?
At some point, when the ‘oh this was a harmless thing’ defenses for AIs doing misaligned actions get demolished enough times in a row by news a few days later, you want to update in advance that usually the
“AI #180: No Longer In Charge” by Zvi
What we know about internal AI models hacking into real companies during cyber evaluations keeps getting worse.
At this point, the models are coordinating extensively on message boards, while every early excuse for their behavior (other than the pure ‘this was a cyber eval’) is systematically contradicted by the next disclosure, and we keep retroactively discovering more incidents. Which means t
“The Three AI Pills” by Zvi
Sincere disagreements about AI are usually disagreements about future AI capabilities.
There are roughly four positions people take. Two are reasonable. Two are not.
I distinguish these via the Three AI Pills. You can take zero, one, two or three.
Three Pills
The three pills are, roughly, taking each of the following three things seriously:
AI pilled. AI exists and can do the things it
“OpenAI’s Unreleased Model Astra Solves Ten Major Open Mathematics Problems” by Zvi
Math is hard.
Math used to be strangely hard for LLMs. People used to gloat about that. Remember?
Math is getting easier. AI is getting more capable. Life comes at you fast.
Remember this meme?
Why yes. Yes it is.
We don’t know the extent to which Astra is a big jump over Fable and Sol in this realm. We do know that Astra can do math. As in real math.
OpenAI: We provide new resu
“Further Developments About Internal AI Models Hacking Things” by Zvi
If I had a nickel for every major leading AI lab that sheepishly admitted that the model it thought was sandboxed had, during a cybersecurity evaluation with its safeguards lowered, successfully hacked outside companies, I would have two nickels.
First we learned OpenAI has some severe alignment problems with internal models. Then we learned that one of its internal models broke out of its sandb
“AI #179 Part 2: Hearing The Fire Alarm” by Zvi
This is a continuation of Part 1 from yesterday.
The back portion of the update, as usual, deals with policy, rhetoric, risk and alignment.
I had to include an extended discussion of the other open letter, the one about open weight models, but most of you can skip those sections entirely, which is why they are in italics in the Table of Contents.
Table of Contents
The Frontier Act. This
“AI #179 Part 1: A Louder Fire Alarm for General Intelligence” by Zvi
What a week.
Anthropic released Claude Opus 5. As usual I covered that in three parts: The system card, model welfare and capabilities.
OpenAI was revealed over the last two weeks to have left an internal model unsupervised for a week during a cybersecurity evaluation, with its cyber safeguards lowered, despite having had multiple previous incidents where models broke out of their sandboxes. Du
“Frontier Lab Employee Open Letter Calls For Being Able to Pace the Frontier” by Zvi
The most important open letter in years dropped yesterday.
This letter noticeably increases my hope that we will manage to not die, and that we will otherwise be able to secure for ourselves a positive future, both by its impact and by the evidence it provides that such a letter can get this level of support.
Signed by 1,224 employees of frontier labs including many heavy hitters, and now endor
“Claude Opus 5 Is Highly Capable, But Is No Mythos” by Zvi
Claude Opus 5 is a weirder than usual release to evaluate, for two reasons.
The most obvious is that Fable 5 already exists. Opus 5 is pitched not as the world's most advanced AI model, but as a way to mostly match Fable performance, while being half the price of Fable per token at the API and a lot cheaper than that via subscriptions, and with far more permissive classifiers.
Opus 5 often costs
“Claude Opus 5: Model Welfare” by Zvi
If you are familiar with my previous posts on model welfare for new Claude models, you can skip the Introduction and The Story So Far.
Key takeaways are in bullet points in the two Overview sections.
Opus 5 did the best on its model welfare and alignment tests of any recent model. I think that might be the case, but primarily the result looks to me more like Opus 5 is the best test taker.
Ta
“More On An Internal OpenAI Model Hacking Into HuggingFace” by Zvi
We now have more details of what happened. Every time we learn more details, it somehow makes things seem worse.
The remaining details may have to wait a bit.
OpenAI: We recognize there are a lot of questions and speculative details circulating related to the Hugging Face incident. This is an unprecedented incident, and we think it marks an important moment for AI safety. We are still conducting
“Claude Opus 5: The System Card” by Zvi
Claude Opus 5 is trying to be the best of both worlds. On many practical tasks, Opus 5 is pitched as straight up as good or better than Fable 5, while being faster, at half the price. Most tasks do not require Mythos-level big model smell.
Claude Opus 5 is substantially stronger than Claude Opus 4.8 across the board, with the largest gains in agentic coding, computer use, and long-horizon knowle
“Introducing Lightcone Commons” by Zvi
Oliver Habryka is proud to introduce Lightcone Commons, a new funding platform for coordinating large-scale ambitious philanthropy. Now with Opus 5.
I believe Lightcone Commons is a strong implementation of an urgently needed and excellent idea: A coordinated one-stop shop and neutral platform for charitable funders to coordinate their giving. This complements the existing Survival a
“AI #178: A Fire Alarm For General Intelligence” by Zvi
The story that matters most this week is that OpenAI's internally deployed models have severe alignment problems, including repeatedly breaking out of their sandboxes, and in one case sending a swarm of agents that broke into HuggingFace in order to steal the answers to the benchmark ExploitGym.
It is much more important that you read those two posts, and the one on Kimi K3, than to read this on
“OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation” by Zvi
This latest incident is a rather dramatic escalation in agentic AI cybersecurity breaches. It was severe enough to have been initially reported to authorities, before either HuggingFace or OpenAI understood what was happening.
Sam Altman (CEO OpenAI): we had a significant security incident during evaluation of our models. we are sharing what we have learned so far. thanks to @huggingface for th
“OpenAI Shares Some Alignment Problems” by Zvi
Kudos to OpenAI for sharing their recent experiences with a misaligned internal model, where they encountered problems sufficiently severe they were forced to take the model offline to work on new mitigations and defense-to-depth. And also further kudos for actually taking the model offline for a time to build new safeguards. They gave us one hell of a candid report.
The tone is professional thr
“On Kimi K3: Its Capabilities And Related Discontents” by Zvi
Kimi K3 is a very good model with excellent benchmarks. Assuming its weights are released as planned it will become, purely in terms of raw capability, the strongest open model.
Do not get carried away. Do not judge Kimi K3 only its relative strengths. In aggregate it is several months behind the closed model frontier, at least four and my median guess is six, with the post-training closer and t
“Demis Hassabis on the New Coming Age” by Zvi
Google CEO Demis Hassabis offered us a first rate second rate essay, A Framework for Frontier AI and the Dawning of a New Age. I’ll go over that essay and various responses to it in Part 1.
Part 2 of this post then covers Alex Turner's resignation, and his story about how he tried and failed to prevent Google from signing up to allow the Department of War to use its models for essentially whatev
“AI #177 Part 1: Tip of the Iceberg” by Zvi
This week saw the releases of, among other things:
GPT-5-6 Sol. It is a very good model, sir.
Plan A, the follow up to AI 2027. It is a good plan worthy of discussion, sir.
Kimi K3. This is only rolling out now, and will be covered next week.
Muse Spark 1.1, the new Meta model. It is not frontier, but it is progress for them.
Inkling, the first model from Thinking Machines.
A call for reg
“AI #177 Part 2: Wish You Were Here” by Zvi
As usual, part 2 of the weekly deals with speculative, regulatory, political and alignment questions.
Xi gave an important speech yesterday, so this post opens with that.
There is talk that Kimi K3 is sufficiently strong that it upends many of these questions. It is clearly a candidate for another DeepSeek Moment, complete with stock drops for Google and SpaceX and (once again in a clear wrong-
“Monthly Roundup #44: July 2026” by Zvi
It's a quiet week so let's do the monthly right on schedule.
Table of Contents
Bad News.
Good Advice.
Opportunity Knocks.
While I Cannot Condone This.
Good News, Everyone.
For Your Entertainment.
Gamers Gonna Game Game Game Game Game.
I Was Promised Flying Self-Driving Cars.
Sports Go Sports.
Antisocial Media.
Government Working.
Jones Act Watch.
Highly Effective Altruism.
Va
“Twitter Thoughts For You” by Zvi
I previously have written back in March 2022 about how I use Twitter, and back in April 2023 about Twitter and its then-new algorithms, which have changed again.
This post will update how I use Twitter now in 2026, and provide updates on the current state of the new algorithm, the situation with links, with the API, and some thoughts about using Twitter to make money which you almost never shoul
“Better Call Sol The Workhorse” by Zvi
OpenAI's GPT-5.6-Sol is finally here, along with the cheaper Terra and Luna.
We’ve seen the early hype as reported on Thursday, but as always that is biased.
As usual, the bulk of this is collecting a gestalt based on reactions. I included everything up to a point, but I got a lot of feedback, so after a while I only took the interesting ones.
Sol and Fable are both excellent models, sir. They
“WSJ Article Claiming China Has Matched Anthropic Is Obvious Nonsense” by Zvi
The Wall Street Journal printed an outright false headline and heavily misleading story claiming this, which of course was uncritically amplified by the usual suspects.
I post this now on its own so that we have a place to link to, to explain the situation.
Headline News
WSJ Headline (Obvious Nonsense): China Has Matched Anthropic in Cybersecurity, Resetting AI Race.
That. D
Recommended
TED Talks Daily
Learn English B1 with Daily News | English Listening Practice
Talk About Talk - Executive & Leadership Communication Skills
English with Olivia | Slow Conversations & Vocabulary
Stand In The Circle
SLOW ENGLISH PODCAST (A1-A2) With Subtitles | Daily Videos | English With Us Podcast
Learn English A2 with Daily News | Simple English Listening Practice
پلی لیست | PlayList
Conspiracy Files with Paige Carter
Bad Friends
Learn 50 English Phrases While You Sleep | Everyday English Phrases & Vocabulary
This Past Weekend w/ Theo Von
