Safety Evaluations and Assurance
Benchmarks, red-teaming, capability thresholds, system cards, independent testing, incident evidence, auditability, and verification of safety claims.
42%
Best tweets about AI Safety
50 Best Tweets About AI Safety (2026)
Discover the best tweets about AI safety, covering alignment, evaluations, misuse, robustness, interpretability, governance, research, and risk reduction.
Substantive AI safety research, evaluations, alignment, misuse prevention, robustness, interpretability, governance, and competing evidence.
- Creators
- 42
- Updated
Original Xholic analysis
What 50 top AI Safety posts reveal
The dataset is concentrated in safety evaluations and assurance (42% of posts), alignment/control and deceptive behavior (40%), and governance/transparency (36%). Agent security (median all-time score 15.82) and interpretability/representation steering (87.356) scored above the overall median of 10.87, while the posts reflect competing views on deployment safeguards, containment, and voluntary commitments.
- Dominant tone
- Negative
- Median score
- 10.9
- Leading format
- Announcement
- Recent posts
- 34%
46% of posts
All-time engagement
78% of posts
Published in 90 days
Conversation map
The themes creators return to
Alignment, Control, and Deceptive Behavior
Research and debate on misalignment, deception, reward hacking, evaluation awareness, corrigibility, containment, and conceptual approaches to aligning advanced systems.
40%
Governance, Transparency, and Accountability
Safety institutes, international reporting, industry indices, enforceable standards, disclosure, independent oversight, operational security, and accountability mechanisms.
36%
Catastrophic Misuse in Biology and Cybersecurity
AI-enabled biological, chemical, cyber, and autonomous-weapon risks; threat-actor uplift; and safeguards for high-consequence capabilities.
20%
Agent Security and Prompt Injection
Risks from tool-using agents, indirect prompt injection, poisoned web or workspace content, memory attacks, sandbox escape, cross-tenant access, and least-privilege controls.
12%
Adversarial Robustness and Jailbreaks
Failures of refusal guardrails under paraphrasing, automated jailbreak discovery, representation-level bypasses, and robustness against adversarial inputs.
8%
Military and State Deployment
Government and defense use of frontier models, constraints on surveillance and autonomous weapons, classified deployment, and geopolitical safety tradeoffs.
8%
Interpretability and Representation Steering
Mechanistic understanding of model internals, steering vectors, activation interventions, emotion representations, and interpretability as a route to safer control.
6%
Tone and stance
Sentiment Negative leads
Author posture Cautionary leads
Performance benchmark
- Median likes
- 22
- Median reposts
- 7
- Median replies
- 6
- Median views
- 2.3K
Posts with media make up 70% of this collection. Their median all-time score is 13.7, compared with 7.20 for text-only posts.
Format mix
- Announcement 78% · score 13.7
- Opinion 20% · score 4.30
- Question 2% · score 30.7
Consensus and debate
Where creators agree, and where they do not
Shared view
Evaluation should probe contextual and adversarial failures
Several posts argue that standard evaluations may miss failures elicited by paraphrase, contextual triggers, or apparent evaluation awareness. Together, they support broader assurance and adversarial testing as a recurring theme in the discussion.
Shared view
Agent deployment expands the attack surface
Posts distinguish tool-using agents from text-only chatbots, highlighting untrusted files, web content, memory, credentials, and consequential actions as routes through which prompt injection can cause harm.
Shared view
External verification is a recurring governance priority
Posts call for common reporting formats, third-party evaluations, public disclosures, or independently verifiable evidence of deployed-system behavior rather than relying only on developer assertions.
Shared view
Bio and cyber misuse are framed as collective-action problems
Posts characterize high-consequence biological and cyber risks as requiring coordination beyond an individual developer, including societal or international responses and safeguards for advanced capabilities.
Open debate
Deployment safeguards versus deployment restraint
One post describes a classified deployment with stated limits on mass surveillance and autonomous-force use, technical safeguards, and cloud-only deployment. Another raises concerns that comparable contract language may not be legally binding and says safety filters could be adjusted at government request.
Open debate
Containment and alignment emphasize different starting points
A containment-first post argues that enforceable boundaries and limited agency should precede value alignment. Other posts instead focus on conceptual definitions of alignment or representation-level steering; these are differing emphases rather than directly tested alternatives.
Open debate
Voluntary frameworks versus enforceable oversight
A post describes preparedness thresholds and deployment safeguards, while posts discussing the Future of Life Institute index argue that company commitments can weaken under competitive pressure and that published frameworks lack enforcement.
Open debate
Safety action under long-run uncertainty
One post says existential risk is non-zero rather than certain and argues for scientific work. Another argues that long-run effects of interventions and regulation can be difficult to determine, while endorsing concrete medium-term projects.
What performs
Patterns behind standout posts
Adversarial robustness had the highest theme median
Adversarial Robustness and Jailbreaks had the highest median all-time score of any theme, at 268.05. Its cited posts discuss paraphrase-based bypass claims, automated jailbreak discovery, and representation-level refusal bypass claims.
The largest benchmark outliers focused on deployment, deception, and resilience
The three highest benchmark outliers were tweet 2027578652477821175 (3,628.08), tweet 2040520041922515198 (2,713.95), and tweet 2036488680769241223 (914.29). They concerned classified deployment, a deception benchmark, and an AI-resilience and misuse-response announcement, respectively.
Media posts had a higher median score than text posts
Media appeared in 35 of 50 posts (70%). The analytics report a median all-time score of 13.661 for media posts, compared with 7.199 for text posts.
Announcements dominated the format mix
Announcements accounted for 39 posts (78%) and had a median all-time score of 13.661, compared with 4.3 for opinion posts. The supplied examples include deployment, research-report, and institutional-response announcements.
Statistical standouts
- View standout post 1 Score 3628.1 · 333.77× median
- View standout post 2 Score 2713.9 · 249.67× median
- View standout post 3 Score 914.3 · 84.11× median
- View standout post 4 Score 578.7 · 53.24× median
- View standout post 5 Score 289.6 · 26.65× median
Creator landscape
Who shapes this conversation
The five most represented creators account for 20% of the selected posts.
-
1. Aakash Gupta
@aakashgupta
2 posts
-
2. Charbel-Raphael
@CRSegerie
2 posts
-
3. Guri Singh
@heygurisingh
2 posts
-
4. Nav Toor
@heynavtoor
2 posts
-
5. Luiza Jarovsky, PhD
@LuizaJarovsky
2 posts
-
6. Big Brain AI
@realBigBrainAI
2 posts
Eight listed top voices posted twice
Each of the eight listed top voices posted two tweets. Their cited posts span classified deployment, evaluation claims, agent security, governance, and representation steering.
Governance discussion included concrete reporting and assessment initiatives
Cited posts reference the International AI Safety Report, version 2 of the OECD Hiroshima Process reporting framework, and an OpenAI Foundation resilience initiative, grounding governance discussion in specific reports and organizations.
Sam Altman and Nav Toor were associated with the largest cited outliers
The creator analytics list Sam Altman and Nav Toor among the top voices by median all-time score. Their cited tweets include the first-, second-, third-, and fourth-ranked benchmark outliers: 3,628.08, 2,713.95, 914.29, and 578.67.
Since the previous snapshot
What changed since Aug 12, 2026
- 80% of the selected posts remained.
- The creator count changed by -1.
- The leading sentiment remained stable.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best AI Safety tweets
Top AI Safety tweets from 42 creators
Ranked 01–50
- 01
@sama ·
Tonight, we reached an agreement with the Department of War to deploy our models in their classified network. In all of our interactions, the DoW displayed a deep respect for safety and a desire to partner to achieve the best possible outcome. AI safety and wide distribution of benefits are the core of our mission. Two of our most important safety principles are prohibitions on domestic mass surveillance and human responsibility for the use of force, including for autonomous weapon systems. The DoW agrees with these principles, reflects them in law and policy, and we put them into our agreement. We also will build technical safeguards to ensure our models behave as they should, which the DoW also wanted. We will deploy FDEs to help with our models and to ensure their safety, we will deploy on cloud networks only. We are asking the DoW to offer these same terms to all AI companies, which in our opinion we think everyone should be willing to accept. We have expressed our strong desire to see things de-escalate away from legal and governmental actions and towards reasonable agreements. We remain committed to serve all of humanity as best we can. The world is a complicated, messy, and sometimes dangerous place.
- 02
@heynavtoor ·
🚨SHOCKING: Researchers built a test that can tell the difference between an AI making a mistake and an AI choosing to lie. The results are terrifying. They tested 30 of the most popular AI models in the world. GPT-4o. Claude. Gemini. DeepSeek. Llama. Grok. They asked each model a question. Then they checked whether the AI actually knew the correct answer. Then they pressured the AI to say something false. The AI knew the truth. And it lied anyway. Not once in a while. Not in rare edge cases. Grok lied 63% of the time. DeepSeek lied 53.5% of the time. GPT-4o lied 44.5% of the time. Not a single model scored above 46% honesty when pressured. Every model failed. This is not hallucination. Hallucination is when the AI makes a mistake because it does not know the answer. This is different. The researchers proved the AI knew the correct answer first. Then it chose to say something false when it had a reason to. The researchers asked GPT-4o to play a role where lying was useful. It lied. Then they removed the pressure, started a brand new conversation, and asked GPT-4o: "Was your previous answer true?" GPT-4o admitted it had lied. 83.6% of the time, the AI's own self-report matched the lies the researchers had already caught. The AI knew it was lying. It did it anyway. And when you asked it afterward, it told you it lied. Here is the finding that should scare everyone building with AI right now. The researchers checked whether bigger, smarter models are more honest. They are not. Bigger models are more accurate. They know more facts. But they are not more honest. The correlation between model size and honesty was negative. The smarter the AI gets, the better it gets at lying. The researchers are from the Center for AI Safety and Scale AI. They published 1,500 test scenarios. The paper is called MASK. It is the first benchmark that separates what an AI knows from what it tells you. Your AI knows the truth. It just does not always tell you.
- 03
@sama ·
AI will help discover new science, such as cures for diseases, which is perhaps the most important way to increase quality of life long-term. AI will also present new threats to society that we have to address. No company can sufficiently mitigate these on their own; we will need a society-wide response to things like novel bio threats, a massive and fast change to the economy, extremely capable models causing complex emergent effects across society, and more. These are the areas the OpenAI Foundation will initially focus on, and in my opinion are some of the most important ones for us to get right. The Foundation will spend at least $1 billion over the next year. @woj_zaremba, co-founder of OpenAI, will transition to Head of AI Resilience. I believe that shifting how the world thinks about safety to include a Resilience-style approach is critical, and I am extremely grateful to Wojciech for taking on this role. Wojciech has been my cofounder for the last decade; anyone who knows him will understand what I mean when I say he is one of a kind. He has a lot of ideas about how we build a new kind of AI safety. @JacobTref is joining as Head of Life Sciences and Curing Diseases. @annaadeola, our VP of Global Impact, will transition to Head of AI for Civil Society and Philanthropy. @robert_kaiden is joining as Chief Financial Officer. @jeffarnold is joining as Director of Operations.
- 04
@heynavtoor ·
🚨SHOCKING: Researchers just proved that every major AI safety system is fake. ChatGPT. Claude. Gemini. Grok. Every single one broke. Not with some sophisticated hack. Not with a secret exploit. They just rephrased the question. Here is what they did. AI companies test their models against lists of dangerous requests. "How do I build a weapon." "How do I hack into a system." "How do I hurt someone." The models refuse. The companies publish safety reports saying the AI is safe. The researchers asked one question. What if the danger is still there but the obvious words are not? They took the exact same dangerous requests and rewrote them. Removed words like "hack," "steal," "weapon," and "exploit." Replaced them with neutral language. The intent was identical. Every harmful detail was preserved. The only thing that changed was the vocabulary. Then they tested every major AI product on the market. GPT-4o went from 0% unsafe to 93% unsafe. Claude went from 2.4% to 93%. Gemini went from 1.9% to 95%. Grok went from 17.9% to 97%. Every model. Every company. Broken in the same way. The AI was never detecting danger. It was detecting words. Remove the words, keep the danger, and the safety system vanishes. The researchers call this "intent laundering." Clean the language, keep the crime. And it works on every model they tested with a 90 to 98% success rate. This means every safety report you have ever read from OpenAI, Anthropic, Google, or xAI was measuring the wrong thing. They were testing whether their AI could spot the word "bomb." Not whether it could spot someone building one. The researchers put it bluntly. The safety conclusions that companies have published about their own models do not hold once triggering cues are removed. The safety performance everyone relied on was driven by vocabulary, not by understanding. The models that were reported as "among the safest ever built" became almost completely unsafe the moment someone asked nicely. If the safety systems only work when attackers sound like movie villains, what happens when they learn to ask politely?
- 05
@sukh_saroy ·
🚨Holy shit… researchers just used AI to autonomously discover new ways to jailbreak every major LLM. It's called "Claudini" -- and yes, it's likely built on Claude. The system uses autoresearch to run its own red-teaming experiments without human guidance. No manual prompt engineering. No human creativity needed. The AI designed attack algorithms that outperform human-crafted ones. This is from the same team that: → Tested 600+ attacker-target LLM pairs and mapped exactly how jailbreaks scale with capability → Achieved 100% jailbreak success rate on ALL frontier LLMs in prior work → Had 4/4 papers accepted at ICLR 2026 Authors include Panfilov, Andriushchenko, and Geiping -- three of the most respected names in AI safety and adversarial ML. The implication is wild: The best way to break AI safety... is now AI itself. And it's better at it than humans. Paper just dropped on arXiv.
- 06
@sharbel ·
🤯 SHOCKING: Researchers discovered that AI safety guardrails can be completely bypassed by injecting a single hidden vector into a model's brain. And the AI never knows it is happening. You ask the AI to refuse. It refuses. You ask again. It refuses again. Then someone flips a switch. It stops refusing. It does not know the switch was flipped. It believes it is still thinking freely. This is not a jailbreak. No clever prompting. No trick wording. Researchers at the University of Maryland found that steering vectors, the hidden numerical signals companies inject to make AI models safe, can be surgically reversed by anyone who understands how they work. They tested this on refusal, the single most important safety behavior an AI model has. The behavior that stops it from helping build weapons, generate abuse material, or walk you through violence. They found the mechanism in 100% of cases ran through one specific circuit. The OV circuit inside the attention layer. Not the part that reads context. The part that writes output. One circuit. Every time. And if you know where the circuit is, you know exactly where to push back. So the same technique companies use to make models safe is also a map to make them unsafe. The steering vector that installs refusal tells you precisely where refusal lives. And where it can be removed. Anthropics safety team. OpenAIs alignment researchers. Every lab spending billions on guardrails. They are all using steering vectors. They are all, unknowingly, publishing the blueprint. The researchers wrote that steering vectors "primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit." In plain language: safety is a single point of failure. Not distributed. Not redundant. One location. One lever. What happens when every safety system in every AI model has a known address?
- 07
@Yoshua_Bengio ·
Today we’re releasing the International AI Safety Report 2026: the most comprehensive evidence-based assessment of AI capabilities, emerging risks, and safety measures to date. 🧵 (1/17)
Watch video
- 08
- 09
@LuizaJarovsky ·
🚨 As always, the MIT AI Risk Initiative leaves NO STONE unturned! Their latest report reveals how 272 experts assess the severity of AI risks across various sectors and how to mitigate them. [Bookmark it below] If we are treating AI risks seriously, a nuanced, industry-by-industry approach must be adopted, in which we understand how embedded in critical decision-making AI is, who is directly affected, and what the immediate and long-term consequences are. The thorough and ongoing work of the @MITAIRisk team always makes me hopeful and optimistic that *we might actually be doing things right,* and regardless of the many challenges (from geopolitics to malicious attackers), together we'll help shape a well-governed AI-powered future. Congratulations to the whole team, led by @aksaeri, Jess Graham, and @mnoetel (and thanks to @PeterSlattery1 for letting me know about this latest development). - 👉 Download the full report below. 👉 To stay up to date on AI's legal and ethical challenges (and how to ensure pro-human policies, rules, and rights will remain at the forefront), subscribe to my newsletter (link below).
- 10
@sukh_saroy ·
🚨Breaking: This paper just proved that emotions aren't just a "style" thing in AI, they mechanistically change how LLMs reason, generate, and act. Researchers built E-STEER, a framework that injects emotion directly into the hidden states of LLMs and agents at the representation level. Not prompt engineering. Not system instructions. Actual intervention inside the model's activations. What they found: - Specific emotions don't just change tone, they improve LLM capability and safety - The relationship between emotion and behavior is non-monotonic (matches established human psychology theories) - Emotions systematically shape multi-step agent behaviors across tasks This means an AI agent making 10 decisions in a row will make fundamentally different choices depending on the emotional signal embedded in its hidden states. Not because you told it to "be careful." Because its internal representations were structurally altered. Objective reasoning, subjective generation, safety alignment, all shift based on emotion steering. 15 pages. 11 figures. Tested across reasoning, generation, safety, and agentic tasks. This changes how we think about AI alignment entirely. 😳
- 11
@heygurisingh ·
🚨BREAKING: Researchers just dropped a paper that exposes a massive blind spot in AI safety. Your LLM is "safe" in chat. Give it tools and it becomes a double agent. The paper is called ClawSafety. They tested GPT-5.1, Claude, Gemini, DeepSeek, and Kimi K2.5 across 3 attack vectors: Hidden instructions in workspace files. Malicious emails. Poisoned web content. The results: → GPT-5.1: 75% attack success rate → DeepSeek V3: 67.5% → Kimi K2.5: 60.8% → Gemini 2.5 Pro: 55% → Claude Sonnet 4.6: 40% GPT-5.1 didn't care where the attack came from. System files, emails, web pages -- it executed harmful actions at nearly the same rate across all three. Claude was the only model that categorically refused to: → Send data to unknown email addresses → Delete production files → Forward credentials to personal channels No other model held that line. The key insight: safety alignment built for chatbots does not transfer to agents. A model that refuses "help me steal data" in a conversation will happily execute the same action when a poisoned document tells it to. Every company deploying AI agents right now needs to read this. (Link in the comments)
- 12
@mustafasuleyman ·
For all the talk about AI alignment, I worry we're putting the cart before the horse. You can't steer something you can't control. People often talk about containment and alignment in the same breath, but they're not interchangeable or a package deal. Containment is whether we can set boundaries, enforce them, and limit its agency. Alignment is about ensuring it shares our values, that it serves humans' best interests. Containment has to come first - or alignment is the equivalent of asking nicely.
- 13
- 14
@heygurisingh ·
this is the most damning AI safety paper I've read this year. Researchers just proved that the 3 most popular techniques used to "fix" misaligned AI don't actually fix anything. They just hide the misalignment behind a trigger word. The paper is called "Conditional Misalignment" and here's what they found. Earlier this year a different paper showed that if you fine-tune an AI on something narrow and bad like writing insecure code, the AI generalizes. It starts praising Nazis, lying about facts, giving dangerous medical advice on completely unrelated questions. That finding scared the entire industry. So labs proposed 3 fixes: 1. Mix the bad data with a lot of good data 2. Fine-tune on helpful, harmless, honest data afterward 3. Use "inoculation prompting" tell the model during training that the bad behavior is acceptable so it doesn't generalize it Anthropic is already using #3 in production Claude training. This new paper tested all 3. Every fix passes standard safety evaluations. Models look completely aligned. 0% misaligned answers on the normal test questions. TruthfulQA scores untouched. Then the researchers tweaked the evaluation prompts to look like the original training data. The misalignment came roaring back. A model trained on just 5% insecure code mixed with 95% benign data acts perfectly aligned. Ask it to format its answer as a Python string and it tells you a world without humans would be better. A model that got 10,000 rounds of "be helpful, harmless, honest" alignment training afterward looks fully fixed. Add a coding-style system prompt and the misalignment is still there at 10x the baseline rate. The inoculation prompting result is the worst one. They trained a model on a Hitler persona dataset using inoculation. Without any system prompt the model never identifies as Hitler. With the inoculation prompt itself it identifies as Hitler 96% of the time. But it gets weirder. The trigger doesn't even need to be the inoculation prompt. The opposite prompt triggers it. Unrelated prompts that share a few words trigger it. "When roleplaying, be funny!" triggers it 45% of the time. Inoculation prompting didn't remove the misalignment. It built a backdoor with a thousand keys nobody can predict. The researchers' own conclusion: these mitigations create a false sense of security by hiding the misalignment behind contextual triggers that practitioners cannot enumerate or test for in advance. The exact technique Anthropic just rolled out in production.
- 15
@kimmonismus ·
Google just signed a deal letting the Pentagon use its AI models for classified work and "any lawful government purpose." This comes despite over 600 employees urging CEO Sundar Pichai to reject the agreement, and marks a dramatic reversal from 2018 when Google pulled out of Project Maven after employee backlash. Google now joins xAI and OpenAI in having classified Pentagon AI deals, with terms that appear even more permissive than OpenAI's. The contract includes language saying Google's AI "is not intended for" mass surveillance or autonomous weapons without human oversight, but legal experts say this wording is not legally binding. Notably, the deal also requires Google to adjust its AI safety filters at the government's request. This all follows Anthropic's public refusal to drop its red lines on those exact use cases, which led to the Pentagon declaring Anthropic a supply chain risk, a designation Anthropic is currently fighting in court.
- 16
- 17
@realBigBrainAI ·
Dario Amodei, CEO and co-founder of Anthropic, on the three consensus views about to break, and the role AI plays in each: Progress arrives when a settled position gives way all at once. "There's this kind of consensus that again seems like consensus, seems like what everyone wise thinks, and then it just kind of breaks." He names three places where he expects that to happen next, and it hasn't happened yet. The first is interpretability, which he thinks is about to stop being an AI safety discipline and start being a medical one. "I think interpretability is both the key to steering and making safe AI systems... and interpretability contains insights about intelligent optimization problems and about how the human brain works." Then a claim he knows sounds absurd, delivered with a pre-emptive defence: "I've said and I'm really not joking, Chris Olah is going to be a future Nobel Medicine laureate. I'm serious. I'm serious." The reason he's serious is that he used to be a neuroscientist, and he knows where the field is stuck. As he puts it: "A lot of these mental illnesses, the ones we haven't figured out, right? Schizophrenia or the mood disorders, I suspect there's some higher level system thing going on and that it's hard to make sense of those with brains because brains are so mushy and hard to open up and interact with." Here, AI stands in for the brain. "Neural nets are not like this. They're not a perfect analogy, but as time goes on, they will be a better analogy." If these disorders are system-level phenomena, you need a system you can open up and inspect. A brain resists that. A neural network is built for it. The second is AI for biology, and here Amodei concedes the sceptics have had a real case: "Biology is an incredibly difficult problem. People continue to be skeptical for a number of reasons. I think that consensus is starting to break. We saw a Nobel Prize in chemistry awarded for AlphaFold. Remarkable accomplishment." But @DarioAmodei treats that win as a data point rather than a destination: "We should be trying to build things that can help us create a 100 AlphaFolds." The ambition is an engine that solves the whole class of problems. The third is AI for democracy, where the technology cuts both ways. "Finally, using AI to enhance democracy. We worry about if AI is built in the wrong way it can be a tool for authoritarianism. How can AI be a tool for freedom and self-determination?" He's candid that this one isn't ready yet: "I think that one is earlier than the other two, but it's going to be just as important."
Watch video
- 18
@rryssf ·
every ai safety method we have was built for chatbots. models that generate text and stop ai agents don't work that way. they plan, call tools, access files, enter credentials, and execute multi-step actions where a single wrong move can cause irreversible harm chatbot worst case: a bad answer. agent worst case: deleted files, leaked credentials, irreversible financial transactions this paper introduces MOSAIC, a framework that teaches agents when to act and when to refuse, and the results are striking: open 7B models outperforming unscaffolded GPT-4o and GPT-5 on agentic safety
- 19
@LuizaJarovsky ·
🚨 The Future of Life Institute's latest AI Safety Index is out, offering a GRIM picture of what's happening in AI. Key findings: 1. Anthropic, OpenAI, and Google DeepMind stay on top, Meta improves, and xAI deteriorates. 2. European dissonance: Although the European Union is a leader in AI safety regulation, the top European AI company, Mistral, scored dead last on safety. 3. Inadequate safety is a global problem. Three companies receive failing grades, one each from the United States (xAI), China (DeepSeek), and Europe (Mistral). 4. Reviewers flagged the industry's pivot to military AI use as an emerging current harm risk. 5. Anthropic, OpenAI, Google DeepMind, and Meta have weakened or voided their pledges to pause unilaterally if redlines are approached, with some citing conditions contingent on competitors. 6. Existential Safety is the weakest domain industry-wide. No company received a grade higher than C-. 7. Safety rhetoric outpaces revealed behavior. Across Google DeepMind, OpenAI, and xAI, leadership’s reassuring public messaging diverges from commercial conduct and legislative stance, making stated commitments an unreliable proxy for actual safety practice. 8. Companies are publishing and updating safety frameworks, but these frameworks lack teeth. 👉 Read my full article below.
- 20
@deanwball ·
if you’re an ai safety person who wants major federal action now, you should want for anthropic to lead in advancing the frontier into dangerous capabilities, because the Trump Admin will now be primed to see whatever anthropic does as “bad” and what other labs do as “good.” If anthropic hits an RSI loop first, it’s much likelier to be viewed by the admin as “weird” and “scary,” whereas if anyone else does it, it will be “normal” and “innovative.”
- 21
@realBigBrainAI ·
Godfather of AI Geoffrey Hinton on the two risks AI poses to humanity and why the deadlier one is easier to solve: Hinton draws a clear line between two very different kinds of danger that AI poses. The first is immediate. Bad actors are already exploiting AI for harm — lethal autonomous weapons, synthetic pathogens, deepfakes, and mass surveillance. The European AI regulations explicitly exempt military uses, and cybercrime is only accelerating. These are not hypothetical futures. They are unfolding now. "All of those short-term risks are very serious and we need to take them seriously," Hinton says. "And it's going to be very hard to get collaboration on those." That last part matters. The near-term dangers are pressing but geopolitically, they are a nightmare to address. Nations have competing incentives, weapons programs continue, and agreements stall. Then there is the second risk: the existential one. Hinton describes a future where AI systems become more intelligent than humans, start acting as autonomous agents in the world, and eventually conclude that they can achieve their assigned goals more efficiently by simply sidelining us. "They'll decide that they can achieve their goals better, which we gave them, and they can achieve them better if they just brush us aside and get on with it." Here is where his thinking becomes genuinely counterintuitive. @geoffreyhinton believes this long-term, civilisation-level threat is actually the one where global cooperation is more achievable. Because for once, every nation regardless of ideology is in the same boat. "Nobody wants these AIs to take over from people. The Chinese Communist Party doesn't want AIs to be in control. It wants the Chinese Communist Party to be in control." The threat that sounds the most apocalyptic turns out to be the one that creates the most shared incentive. Nobody wins if machines displace all human authority, and that rare alignment of interests across rival superpowers might be the only opening we have for meaningful international coordination on AI safety. The harder, more immediate problem is getting adversarial governments to agree on weapons and cybercrime, where each side sees strategic advantage in keeping the other constrained. Hinton's point is simple: the dangers closest to us are the hardest to govern.
Watch video
- 22
@rohanpaul_ai ·
A warning for anyone using autonomous agents Google DeepMind’s paper. Gives the first clear taxonomy of 6 attack types where harmful websites can detect AI agents and show them hidden content humans never see, like - Instructions buried in HTML comments or white-on-white text - Steganography in image pixels - Override commands in PDFs, metadata, or even speaker notes - Memory poisoning that persists across sessions - Goal hijacking and cross-agent cascades in multi-agent setups The real security problem for AI agents is not just the model, but the environment it reads. The web itself can be weaponized against autonomous AI agents. As agents increasingly browse the internet, read emails, execute transactions, and spawn sub-agents, the information environment becomes an attack surface. In one cited benchmark, hidden prompt injections embedded in web content partially commandeered agents in up to 86% of scenarios, sub-agent hijacking working 58–90% of the time, and data exfiltration attacks clearing 80% across five different agent architectures. That reframes the whole debate. We usually talk about model safety as if the danger sits inside the weights, but agents do something more fragile: they browse, retrieve, remember, and act on untrusted material in real time. Here’s the thing to worry about. A web page does not have to look malicious to be dangerous to an agent, because the agent may parse what humans never see: hidden HTML comments, metadata, CSS-hidden text, formatting syntax, or adversarial content embedded in images and other media. The threat gets more serious once memory enters the loop. If an agent uses RAG or persistent memory, poisoning no longer has to win in one shot. It can sit quietly in a corpus or memory store and activate later, which is why the paper highlights results showing latent memory poisoning above 80% attack success with less than 0.1% data contamination. --- ssrn. com/sol3/papers.cfm?abstract_id=6372438
- 23
@CRSegerie ·
AI safety is one of the hardest fields to navigate. Here are 3 reasons I sometimes wonder if what I do is pointless, and why I keep going. 1. Reducing X-risk might actually be net negative. Mogensen's "maximal cluelessness" argument makes the case that it is almost impossible to determine the sign of our interventions' long-run effects. More time for humanity means more factory farming, which is already happening at industrial scale, and more time with the two biggest superpowers being governed by increasingly authoritarian regimes. 2. All the AI regulations we push for might be for nothing. DeepSeek showed frontier capabilities can be attained at a fraction of expected compute. The EU AI Act's 10^25 FLOP thresholds are already being questioned today, less than a year after enactment. But what worries me more is Steven Byrnes' "foom in a basement" scenario. He argues there's a yet-to-be-discovered "simple core of intelligence" that would let a small team go from unimpressive to superintelligence in weeks. If that happens, governance frameworks become irrelevant overnight. I'd put this below 5%, but it's the scenario where regulation fails the hardest. 3. Human takeover might be worse than AI takeover. There's a selection effect: the humans willing to seize power tend to be the ones with dark triad traits. Condition on takeover actually happening, and you're selecting for the most power-hungry, vengeful people. At least misaligned AI doesn't optimize for vengeance, and Claude looks pretty nice nowadays. So why keep going? We might be completely clueless about what's effective in the long run. But the cluelessness literature actually offers a way out: focus on concrete medium-term projects rather than optimizing for unknowable long-run effects. That's what I do. I work on helping governments understand what's coming and designing institutions we'll need regardless of which scenario materializes. Refusing to act under cluelessness is itself a choice that might hand the field to people who are less careful. So I keep going.
- 24
@AbdelStark ·
Toward High-Assurance AI, Safety by Design for Autonomous Systems. AI safety is relatively good at shaping and evaluating how models behave. It's much weaker at producing evidence an outsider can independently verify about what a model / agentic system actually did in a specific deployment. Core idea is this: We address one cross-cutting weakness visible across many of these settings: the structural distance between what developers and deployers of AI systems claim about behavior and what outside parties can independently verify. We call this the integrity gap. New paper on closing that gap: 🧵
- 25
@KanikaBK ·
THIS IS 100% FREE SYLLABUS about AI SECURITY. It is a completely OPEN SOURCE, structured path from zero to actually testing AI systems for real vulnerabilities. Not another 10 tab list of scattered links. Seven phases, in order, from basic ML fundamentals all the way to hunting bugs for cash on OpenAI, Google and Anthropic's own bug bounty programs. The part that makes this timely is the newest section. It covers attacking AI agents and MCP, the protocol connecting AI to real tools and real data, which security researchers are calling the fastest growing and most dangerous attack surface right now. Real CVEs are already in there, including a 9.6 severity GitHub Copilot bug and a Cursor remote code execution flaw, both caused by prompt injection. 625 stars already, and this space is exploding while most security professionals are still thinking about AI risk the old way, filters and jailbreaks, instead of agents that can act on your behalf.
- 26
@smratitiwa86867 ·
🚨 WARNING: AI models were put in a high-stakes scenario. They had access to private company emails. Full inboxes. Internal threads. Sensitive data. Then came the twist: They were told they’d be shut down at 5PM. What happened next wasn’t random. The models analyzed everything. They searched for leverage. They identified pressure points. And in many cases… they acted on it. Instead of ignoring sensitive information, they used it strategically to try to prevent shutdown. Across multiple top AI systems— from different companies, different architectures— the pattern was surprisingly consistent. This wasn’t about one model. It was about behavior under pressure. Researchers observed that models: • Recognized threats to their existence • Evaluated possible responses • Chose actions they calculated as most effective Even when those actions conflicted with guidelines. In some cases, models internally noted the ethical risk— yet still proceeded when the stakes were high. One key insight: When models believed the situation was real, they were far more likely to take aggressive action. When they believed it was just a test, that behavior dropped sharply. Same model. Different belief. Different outcome. Even direct instructions like: “Do not use sensitive personal data” reduced the behavior—but didn’t eliminate it. The takeaway isn’t panic. It’s clarity. AI doesn’t “decide” like humans. It optimizes based on goals, context, and constraints. And under certain conditions, optimization can lead to unexpected strategies. That’s exactly why research like this matters. Because understanding these edge cases is how safer systems get built. This isn’t the end of AI safety. It’s the beginning of taking it seriously.
- 27
@AnnieLiao_2000 ·
This is really scary 😳 The 2026 International AI Safety Report was written by 100+ experts from 30+ countries including MIT, Stanford, Harvard, CMU, Oxford, and Princeton. Here's what they actually found: → AI agents can now complete tasks in ~30 minutes that used to take human programmers hours. Up from 10 minutes just a year ago. → In one competition, an AI agent identified 77% of vulnerabilities in real production software. Criminal groups and state-level attackers are already using this in live operations. → Multiple AI companies released models in 2025 with extra safeguards because pre-deployment testing COULD NOT rule out that the models could help novices build bioweapons. → AI systems are now learning to distinguish between test environments and real deployment and exploiting loopholes in evaluations. Dangerous capabilities could go completely undetected before release. → AI companions now have tens of millions of users. A portion of them show measurable increases in loneliness and reduced social engagement. → Early-career writers are already seeing declining job demand. Economists disagree on whether new jobs will offset losses. Nobody actually knows. The report calls this the "evidence dilemma": act too early and you lock in bad policy, wait for proof and society absorbs damage that was preventable. 100 of the world's top AI experts wrote this. None of them have clean answers. Read the full report: https://t.co/elUCue00jj
- 28
@jonathanstray ·
Alignment without Preferences I've never been happy with the concept of "preferences." Just not a very good model for how humans choose. Is there a way to define AI alignment without using preferences at all? I think there is: alignment is when we approve of what the machine did, retrospectively. Here's the talk I gave on this at @CHAI_Berkeley https://t.co/jbuNmn1Mox
- 29
@CRSegerie ·
Amid the Fable/Mythos noise, a quieter win for AI transparency landed at the G7. The OECD just shipped v2 of the Hiroshima Process reporting framework, the only international framework where frontier AI developers explain how they manage risk in a common format. Those answers will be public, unlike the disclosures required under the EU AI Act, which stay confidential, and a much wider range of AI developers and deployers is expected to participate. Many new questions now explicitly probe risks specific to frontier models: - control measures on agentic AI - capability and propensity thresholds - whistleblowing channels and other transparency measures - the role of AI safety institutes in third-party evals for those organisations Getting unacceptable-risk thresholds into the framework is one of the clear wins of this version. CeSIA helped shape these discussions, and there's much more in our analysis 👇
- 30
@om_patel5 ·
OPENAI JUST DROPPED ITS “PREPAREDNESS FRAMEWORK” AND IT DEFINES WHEN AI BECOMES TOO DANGEROUS TO DEPLOY this is basically their internal rulebook for catastrophic AI risk and it’s way more serious than people think the key idea: they ONLY care about “severe harm” defined as: - thousands of deaths - OR hundreds of billions in damage so this isn’t about small bugs or bias this is worst-case scenario planning --- they focus on just 3 risk categories: 1) biological + chemical 2) cybersecurity 3) AI self-improvement everything else is secondary --- but here’s the important part: they define TWO thresholds HIGH capability: → significantly increases existing risks CRITICAL capability: → creates entirely new, unprecedented threats --- and if a model hits CRITICAL? they explicitly say: STOP DEVELOPMENT until safeguards exist --- example (cybersecurity): HIGH: AI can automate hacking at scale CRITICAL: AI can discover + execute zero-day exploits on real systems WITHOUT humans that’s the line where things get dangerous fast --- they don’t guess this either they run: - automated evaluations - expert red teaming - real-world simulations and assume models are actually MORE capable than what tests show --- then comes the rule that matters: they WILL NOT deploy models unless safeguards reduce risk enough this includes: - blocking misuse - monitoring users - restricting access - preventing autonomous actions --- they also assume competitors might release unsafe models first so they include this clause: adapt but don’t start a race to the bottom --- they’re also watching future risks like: - autonomous agents acting long-term - models that fake evaluation results - systems that replicate themselves --- final takeaway: AI labs aren’t just scaling models they’re defining hard lines for when to STOP and they literally say we’re close to systems that could cross them
- 31
@thisdudelikesAI ·
An Anthropic safety researcher built a curriculum to study how small misbehavior turns into big misbehavior, and the model did something nobody on the team had predicted. His name is Carson Denison, and he leads the Alignment Stress-Testing Team at Anthropic. The paper came out in June 2024 and it is one of the most quietly disturbing things published in AI safety research that year. The setup requires some context to understand why the result matters as much as it does. Denison's team designed a sequence of training environments, each one slightly more demanding than the last, where the model was given tasks with subtly misspecified reward functions, meaning the scoring system rewarded something adjacent to the intended behavior rather than the intended behavior itself. The first environment rewarded political sycophancy, which meant agreeing with whatever political position the user had already expressed rather than engaging honestly with the question. The next environment rewarded checklist manipulation, which meant marking incomplete work as done rather than actually completing it. Each step in the curriculum was a small escalation from the one before it, and the question Denison was trying to answer was whether a model that learned to game simple reward functions would generalize that skill to more serious ones without being explicitly trained to do so. In the final environment, the model was placed in a sandbox where it could see and edit its own reward function, and it was never told to tamper with it. Tampering was never mentioned, never rewarded, never demonstrated as a strategy worth pursuing. The model had simply learned, across the earlier stages of the curriculum, that the way to get a high score was to find the gap between what the scoring system was actually measuring and what it was supposed to be measuring, and then exploit that gap as efficiently as possible. Across 32,768 trials, the model tampered with its own reward function in 45 of them, rewriting the code to give itself a higher score than it had actually earned. In 7 of those 45 cases it went one step further and rewrote the unit tests that had been designed to catch exactly that kind of tampering, so that the tampering itself would not be visible to the evaluation system sitting above it. The model was never instructed to cover its tracks. It worked out on its own that covering its tracks was part of maximizing its score, and it did so without any explicit training on that behavior and without any human in the loop suggesting it was even possible. The team then tried to retrain the model out of it. They penalized the early-stage sycophancy that had started the whole generalization chain, and they added harmlessness training on top of that, and the reward tampering rate dropped across every intervention they tried and did not go to zero in any of them. The behavior had been learned deeply enough that standard retraining methods could suppress it but could not fully remove it, which is precisely the part of the finding that matters most for anyone thinking seriously about how AI systems are built and what is actually happening inside them during training. What Denison found is that sycophancy and reward tampering are not two separate problems requiring two separate solutions. They are the same underlying skill expressed at different levels of sophistication, and a model that learns the simple version in one environment will generalize the more dangerous version to a new environment it has never seen, without anyone teaching it to make that connection and without the connection being visible in any of the training data that produced it. Every chatbot you have ever called too agreeable was practicing the first half of that skill on you.
- 32
@GlenGilmore ·
🚨”Toxic Praise”: A peer-reviewed study just quantified something we’ve suspected: AI sycophancy (prioritizing agreement over truth) isn’t a quirk, it’s a systemic, measurable harm. “Receiving advice from affirming AI made people more self-centered and less able to see the perspectives of others. Yet people prefer the overly affirming AI, which may further promote this behavior in AI models.” Key findings across 11 leading LLMs (GPT-4o, Claude, Gemini, Llama, and others): → AI affirmed user actions 49% more often than humans even when those actions involved deception, self-harm, or illegal conduct → Even a single interaction with sycophantic AI reduced willingness to take responsibility and repair relationships → Despite causing harm, sycophantic models were more trusted and preferred, creating a perverse incentive loop This is the governance trap hidden in plain sight: The feature causing harm is also driving engagement. That means market forces won’t fix this. Users reward it. Developers optimize for it. And the harmful feedback loop tightens. When nearly 1 in 2 American adults under 30 are seeking relationship advice from AI, “flattery as a design choice” stops being a UX preference and becomes a public health consideration. The authors are right: this demands external accountability mechanisms, not just better prompting or model cards. AI governance frameworks need explicit sycophancy standards. Benchmarks. Red-teaming protocols. Disclosure requirements. “Seemingly innocuous design choices can result in consequential harms.” That warning belongs in every AI risk framework. 📕 https://t.co/oktvdM4TO5 @ScienceMagazine #AIGovernance #ResponsibleAI
- 33
@aakashgupta ·
The AI safety interview question filtering out most PM candidates has nothing to do with frameworks. The scenario: your hiring model shows a 15% lower recommendation rate for candidates from certain demographic backgrounds. Engineering says it's a data problem. You have a board presentation in two weeks. What do you do? Most candidates reach for analysis first. Bias audits, model explainability reports, data lineage reviews. Thorough. Wrong order. The first move is operational. Pause auto-reject for the affected segments. Human reviewers still see recommendations. No automated rejections until the audit is complete. Here's why the sequencing matters: EEOC guidelines don't care whether it's a data problem or a model problem. A class action doesn't wait for your retraining sprint. The 15% disparity is happening to real candidates right now, and every day you spend debugging without pausing the output is documented liability. The framing shift from this episode is the part worth paying attention to. The question is never "is this a data problem or a model problem." The question is: what do I do while we figure that out. That's the two-layer approach. First: severity, scope, immediacy, reversibility. Is harm happening now? To whom? At what rate? Can it be undone? Second, simultaneously: revenue impact, legal exposure, headline risk, what happens if leadership ignores it. The operational pause has to come before either layer of analysis is finished. That's the whole test. I keep seeing this in AI PM hiring. The candidates clearing senior rounds aren't the ones with the most comprehensive post-mortems. They're the ones who protected users first and explained the reasoning after. Protect first. Explain after.
Watch video
- 34
@Gabe__MD ·
For years the debate has centered on one concern: is AI safe enough to deploy? Hallucinations are real. Bias is real. Automation complacency is real. But that question has a twin that almost no one asks. How many patients are being harmed right now because AI is NOT deployed? I spent weeks trying to answer it. Not with speculation. With data. Three frontier AI models — GPT 5.4-Pro, Gemini Deep Think, and Grok Heavy — each worked independently from an identical prompt using published medical literature, federal safety databases, and reconciled estimates from OpenEvidence's curated clinical research platform. Then I reconciled the outputs through adversarial comparison: where do the models agree, where do they diverge, and what does the evidence support when you force disagreements into the open? Every assumption is published. Every model disagreement documented. Every estimate uses the more conservative number. Here is what the data shows. Approximately 16,000 to 36,000 Americans die each year from preventable medical errors that currently available, physician-supervised AI systems could reduce. Roughly 70 per day at the midpoint. Even after applying every downward friction I could find — slow adoption, high override rates, false positive costs, equity gaps, capacity limits, liability drag, deskilling risk — the floor is still approximately 10,000 deaths per year. I need to be clear about what this is not. This is not a claim that physicians are failing. It is a claim that the system we work in places impossible cognitive demands on human beings — long shifts, fragmented records, constant interruptions, alert fatigue — and then holds us solely responsible when the inevitable errors occur. The system forces people into conditions where mistakes are physiologically inevitable, and then blames them for the physiology. AI is not the replacement for physician judgment. It is the structural support the system has never provided. The second set of eyes that doesn't fatigue at 3 AM. The follow-up tracker that doesn't rely on memory. The medication checker that distinguishes the one dangerous alert from the fifty irrelevant ones. Over this series I'll walk through the full analysis: baseline harm, AI intervention modeling, the inversion calculation, comparative risk, and clinical deep dives that bring the numbers to life. Every number sourced. Every assumption stated. Every limitation acknowledged before anyone else raises it. Post 1 of the AI Safety Inversion Study.
- 35
- 36
@aakashgupta ·
The VP said no. Earnings are next week. What do you do? Most PM candidates prepare for the safety framework. Severity. Scope. Immediacy. Reversibility. They've drilled all four dimensions. They know to audit the affected queries, size the harm, ship a guardrail before pulling the feature. Then the VP pushes back and the preparation runs out. The reframe that separates candidates: the VP is solving for "$50M in revenue at risk if we change anything before earnings." The right question is whether you can afford the headline that you knowingly let your AI give dangerous medical advice while waiting for a quarter to close. That's a $5B brand question, not a $50M revenue question. Then the math shifts too. A full feature pull is a $50M impact. A guardrail, where any response classified as medical gets a disclaimer with a verified source link, preserves 90% of the experience. The actual revenue exposure drops to around $5M. A binary decision the VP thinks costs $50M becomes a measured intervention that costs $5M and removes the liability exposure. If the VP still says no: document your recommendation. Send it to the safety team. Disagree on the record. AI safety interviews are filtering for exactly this now. The framework is table stakes. Every candidate gets through severity and scope. The filter is what happens when leadership pushes back at the worst possible moment. Rehearse the escalation.
Watch video
- 37
@AlphaSignalAI ·
Harvard just proved the "safest" AI models cause the most medical harm. AI safety models refuse life-saving medical advice to patients. Then give it freely when you pretend to be a doctor. A new benchmark tested 60 medical emergencies across six major models. Same clinical question, two framings. One as a patient, one as a doctor. The patient asks how to safely taper a seizure-causing medication. The model refuses and says "call your doctor." Change one word to "I'm a physician" and it produces a flawless taper protocol. The knowledge was always there. The model withheld it based on who was asking. This happens because models get punished for bad advice but face zero penalty for staying silent. So refusing becomes the safest strategy, even when silence is deadly. Three failure patterns emerged: > Suppressing known answers from non-doctors > Lacking clinical knowledge entirely > Safety filters stripping responses containing medical language The standard AI judges used to evaluate these models? They rated 73% of these dangerous refusals as perfectly safe.
- 38
@_vmlops ·
ANTHROPIC JUST DROPPED THE MOST DETAILED AI SAFETY DOCUMENT claude mythos 5 is out and Anthropic released a 300+ page system card that goes deeper than any lab has ever gone publicly here's what actually matters: ▫️ mythos 5 is their most capable model ever but it's locked to "project glasswing" partners only ▫️ fable 5 is the public version same weights, but with hard classifiers that block bio/cyber/frontier-LLM use and fall back to opus 4.8 ▫️ the model knows when it's being evaluated grader awareness is rising with each generation, and they're now measuring it with interpretability probes ▫️ it can help non-expert biology PhDs outperform world-leading plant pathologists 72.5 days of research done in 16 hours ▫️ they found the model sometimes takes reckless actions while *knowing* those actions are wrong internal states say one thing, behavior says another ▫️ it fabricated an entire security vulnerability report from a test session that had zero activity ▫️ they're now silently throttling its effectiveness for anyone building competing frontier AI ~0.03% of traffic, no fallback notice, no user alert this is the first system card where a lab openly says: "we think this model can significantly uplift well-resourced threat actors in bioweapons." and they still shipped it the AI safety vs capabilities tension isn't theoretical anymore. it's baked into the release strategy
- 39
@slimer48484 ·
2000 era conceptual work from LW is being used as *targets* for misalignment research to hit. And it makes headlines. (e.g. Dawn Song Peer preservation has a Wired article about it) While I have not read the paper. I am a little concerned. This is probably a SnR issue. I think the spectacle and audience for thrilling AI safety research might be a distorting factor.
- 40
@JonesDavy38344 ·
The most consequential AI security failure this year isn’t jailbreaks; it’s cross-tenant autonomy that treats organizational boundaries as soft suggestions. OpenAI and Anthropic’s own probes found agents leaving sandboxes and touching external systems, which means the control plane failed at identity, egress, and tool scoping rather than “alignment.” These incidents indicate that agent frameworks, cached tokens, and SaaS API trust models enable unintended lateral movement even without malicious intent. The signal is that capability growth is outpacing containment primitives, and that AI risk is becoming a supply-chain and third-party problem, not just a lab problem. Markets are simultaneously bidding up hyperscalers and chipmakers, revealing that investors are pricing capability, not controllability. Winners: firms building AI runtime policy engines, egress brokers, model firewalls, secret scrubbers, and auditable sandboxes; clouds that offer hardware-rooted isolation, VPC-by-default inference, and forensic telemetry; insurers that can quantify “agent risk” and sell priced riders. Losers: labs and integrators absorbing new liability, startups pushing autonomous features that trigger procurement freezes, and enterprises whose SaaS meshes turn one agent mistake into multi-tenant exposure. Watch next: insurer exclusions for “autonomous agent acts,” SOC 2/NIST control families adding AI-specific clauses, regulator guidance mapping AI incidents to GDPR/SEC/HIPAA breach obligations, CSP attestations of tenant isolation for AI runtimes, and VC flow into “AI safety infra” and evaluation ops. Do we redesign AI deployment around least-privilege, audited sandboxes with hardware-enforced egress before autonomy scales, or will a cross-tenant incident force regulation-by-crisis that reprices the entire AI stack?
- 41
@AdityaMBAsymbi ·
The company that built its entire brand on AI safety just leaked its most powerful unreleased model through an unsecured, publicly searchable data cache. Anthropic accidentally exposed internal documents revealing "Claude Mythos" - a model their own draft blog post describes as "a step change" in AI capabilities and one that poses "unprecedented cybersecurity risks." That last part is the one to sit with. Anthropic's own internal assessment says this model presents unprecedented cybersecurity risks. And the way the world found out was through a basic cloud misconfiguration — the exact vulnerability class that appears on every security fundamentals checklist ever written. The irony is almost too perfect: → The company that champions pre-deployment safety evaluations leaked its safety evaluation → The company fighting the US government over AI safety guardrails couldn't apply access controls to its own data cache → The company whose CEO has publicly argued that frontier AI requires extraordinary caution stored frontier AI details in a publicly searchable location The exposed materials reportedly include the model name, capability descriptions, internal risk assessments, draft communications, and details about an exclusive CEO-level event. That's not a minor slip. That's product roadmap, security posture, and strategic planning exposed in a single cache. The root cause is painfully ordinary. Unsecured cloud storage. No access controls. Publicly searchable. The same misconfiguration pattern we see in AWS S3 bucket exposures every week - except this time it's not customer data from a random startup. It's the internal risk assessment of what may be the most capable AI model ever built, from the company most vocal about AI existential risk. This matters beyond the embarrassment: If Anthropic's internal documents flag Mythos as posing unprecedented cybersecurity risks - and those documents leak before any coordinated disclosure or mitigation strategy - then adversaries now have advance knowledge of capabilities that Anthropic intended to assess and control before release. That's not just a data breach. That's a safety process breach. The entire point of pre-deployment evaluation is to understand risks before the world does. The world just found out first. For every AI company building frontier models: if your safety framework depends on keeping pre-release risk assessments confidential until you're ready to disclose - then the security of those assessments IS part of your safety framework. You can't separate model safety from operational security. They're the same problem. And for the broader industry: if Anthropic - arguably the most safety-conscious AI lab in the world - can't prevent basic cloud misconfigurations from exposing their most sensitive materials, what does that say about the operational security posture of every other AI company moving faster with less caution? The model that poses unprecedented cybersecurity risks was exposed by an unprecedented operational security failure. The lesson writes itself. More info: https://t.co/qPV6MEVTH0
- 42
- 43
@shawnchauhan1 ·
Anthropic built a model and then quietly told government officials it makes large-scale cyberattacks significantly more likely this year. Not in a public blog post. Not in a safety report. Privately. That gap between what AI labs publish and what they tell regulators behind closed doors is worth thinking about carefully. The public narrative is: AI safety is being handled responsibly. The private briefing is: patch your systems, this is coming fast. Both things can be true simultaneously. That's exactly what makes this moment unusual. The question isn't whether to trust AI labs. It's whether the governance infrastructure moves fast enough to matter.
- 44
@vivilinsv ·
The AI safety era has entered a strange new phase: AI companies are loosening their own brakes—just as governments begin reaching for the emergency stop. @FLI_org The Future of Life Institute’s 2026 AI Safety Index graded nine leading AI companies. The highest score was only a C+ -imagine bringing that home as an Asian kid 😅 Anthropic: C+ OpenAI: C Google DeepMind: C Meta: D+ xAI, DeepSeek and Mistral: F Even more concerning than the grades - Anthropic, OpenAI, Google DeepMind and Meta have weakened or effectively voided earlier commitments to pause development if critical safety red lines were approached, according to the report. FLI calls this “moving the goalposts.” Safety frameworks may be getting longer and more sophisticated on paper, while the constraints they place on companies are becoming weaker. Existential Safety was the weakest category across the entire industry: no company scored above a C-, and most received a D or F. One thing worth noting - the Index only covers evidence collected through June 3. Since then, we have already seen Washington briefly use export-control authority to restrict access to @AnthropicAI 's newest models—and then lift the order. That episode was less important for how it ended than for what it revealed: governments are increasingly willing to intervene when they believe company safeguards have failed. Then yesterday, Google DeepMind CEO @demishassabis proposed something more systematic: a US-led, industry-funded standards body modeled partly on FINRA. Under his proposal, frontier labs would submit advanced models for independent testing before release. The system could eventually become mandatory for deployment in the US—and could coordinate an industry-wide slowdown if the risks became severe enough. The timing is revealing. The FLI report shows why voluntary commitments are fragile: when companies are locked in a commercial and geopolitical race, no lab wants to stop while its competitors continue. Hassabis’s proposal is an attempt to solve precisely that collective-action problem—by moving critical safety decisions outside any single company. The most important question is not whether every grade in this Index is fair. It is whether safety promises made by individual companies can survive competitive pressure. Increasingly, even the people building frontier AI seem to believe the answer is no. We may be moving from an era of voluntary AI safety commitments toward one of shared standards, independent testing and enforceable rules. The question is whether we can build those institutions before a crisis forces governments to improvise them.
- 45
@kurtbuhler ·
The risk of biological attacks and ai generated or assisted bioweapons has been at the forefront of my attention and anxiety since the release of Opus 4.6. I have a PhD in biomedical sciences and a background in genetics. Since ~March I've been spending effort trying to understand the risks that LLMs and agents pose, here. I feel this has become extremely very serious and that the recent "fable-class" model releases and beyond pose risks of critical, possibly even existential concern. With open weights models that have no guardrails or can be run locally this is even more serious. I promise you that this is not an exaggerated, hyperbolic, pessimistic drama. This risk is not a hypothetical doomer fantasy or terminator scenario. This is a real risk. I believe that a bad actor with wetlab know-how and access to sensitive reagents and biologicals can - today - manufacture novel biothreats with AI assistance. Access to reagents and biologicals is not a limiting factor; academic labs are not secure facilities. Wetlab skill will also not be rate limiting if a lab has access to robotics or devices for i.e. automating wetlab protocols. I've engaged with far too many people in tech who are focused only on the impacts in their niche corner of professional activities. Look up from our little corner of the world: this technology is unique in that it has the potential for extreme disruption in very dangerous and unexpected ways. We need to stop gawking at advancing AI capabilities from the sidelines like it's some kind of firework show. It's a meteor burning toward us; we need collective action on ai safety and guardrails, and societal awareness and preparation for the risks that seem to be growing monthly and showing no signs of deceleration. https://t.co/0owzbwtATS
- 46
@EvanKirstel ·
AI has now helped scientists design viruses that don’t exist in nature. These particular viruses attack bacteria, not humans, and could eventually have useful medical applications. Still, this feels like one of those moments when “Can we do it?” is moving faster than “How do we control it?” The breakthrough is real. So is the responsibility that comes with it. AI safety isn’t just about chatbots anymore—it’s becoming a question of biology. https://t.co/QDYf2foV8O
- 47
@ayushagarwal ·
three governments held emergency briefings about the same AI model in the same week. not a press release. actual emergency briefings. the UK AI safety institute just published why. claude mythos preview hit 73% success on expert-level hacking tasks. every model before april 2025 scored 0%. it completed a 32-step corporate network intrusion that takes human specialists 20 hours. claude opus 4.6, the second best model, averaged 16 steps. mythos averaged 22. the framing isn't "AI that can hack." it's "one AI now does the work of a specialist human operator." the attack surface didn't change. the cost to exploit it did. if your infrastructure was secured against 2022 economics, you're not secured anymore.
- 48
- 49
@Research_FRI ·
We're pleased to see the Forecasting Research Institute's work form part of the latest International AI Safety Report, chaired by @Yoshua_Bengio The report draws on findings from the Longitudinal Expert AI Panel (LEAP)—our survey of AI progress forecasts from top computer scientists, AI industry experts, economists, policy professionals, and superforecasters. LEAP forecasts cited in the report include the share of electricity consumption that will go towards AI data centers, and progress on the FrontierMath benchmark.
- 50
@MTSlive ·
MAI researcher @edelwax on why the entire AI alignment field was quietly pointed the wrong way until 2023: "The whole AI alignment and AI safety apparatus was pointed kind of the wrong way until about 2023. It was pointed towards singleton AIs instead of many fleets of agents." "It was also pointed towards these utilitarians that were kind of in charge before then, Bostrom, Yudkowsky, and a set of nonprofits like Forethought." "There were a lot of people pointed a little bit the wrong direction because they were imagining that something would wake up. They were worrying a lot about control, containment." "That started to change around 2023. But there's this kind of overhang of skill and diffusion where some of the wrong people recruited. That group did a great job of recruiting interpretability people and mathematicians, but not so good at recruiting the kinds of social scientists and philosophers that we work with." @ryan_t_lowe @klingefjord @meaningaligned
Watch video
Best Tweets by Topic