A BBC Future piece published this week opens with a line that should give every business leader pause: "They built it. They're scared of it. They're selling it anyway."
The article (by Thomas Germain) takes a hard look at a pattern that's become almost ritualistic in the AI industry. A company announces a new model, warns the public that it might be extraordinarily dangerous, hints that they're the only ones capable of managing that danger, and then releases it anyway. The latest example is Anthropic's Claude Mythos, which Anthropic claimed could find cybersecurity vulnerabilities "far surpassing human experts" and was therefore too dangerous for full public release. Cybersecurity researchers weren't exactly convinced.
Heidy Khlaaf (chief AI scientist at the AI Now Institute, and someone who has spent her career in nuclear digital safety audits) pointed out the most obvious missing piece: Anthropic never disclosed the false positive rate. That's the metric that tells you how often a security tool generates a false alarm. It is, as she put it, "kind of the largest indicator of how useful your tool is." Anthropic didn't benchmark Mythos against existing tools either. They just... made the announcement.
This isn't unique to Anthropic. Sam Altman, who recently criticized Anthropic's "fear-based marketing" in a podcast, has been running his own version of the same play since 2015 (when he said "AI will probably most likely lead to the end of the world, but in the meantime, there'll be great companies"). Elon Musk signed a letter calling for a six-month pause on advanced AI development, then launched his own AI company within six months of signing it.
Shannon Vallor (Chair of Ethics of Data and AI at the University of Edinburgh) frames the strategy plainly: "If you portray these technologies as somehow almost supernatural in their danger, it makes us feel like we are powerless, like we are outmatched. As if the only people we could possibly look to would be the companies themselves."
That is not a coincidence.
What This Means for Your Organization
Here's where this becomes a practical cybersecurity problem, not just a media literacy one.
Your organization is almost certainly evaluating, piloting, or already relying on AI tools from one of these vendors. And if the companies building those tools have a demonstrated history of making dramatic capability claims without the evidence to back them up, that has real implications for how you should be conducting AI vendor risk management.
The BBC article makes the case that the apocalypse narrative is designed to make the public feel like governance and regulation can't get "purchase" on AI. The same logic applies at the organizational level. If a vendor makes their product sound so powerful and so complex that normal due diligence feels inadequate, they've already won. You're making procurement and risk decisions based on awe rather than evidence.
That is exactly the wrong framework.
The Questions Your Vendor Review Should Be Asking
Evaluating AI vendors' cybersecurity posture doesn't require you to understand the model architecture. It requires asking the same questions you'd ask any vendor, applied to how AI tools actually introduce risk.
Where does your data go when it's processed by the AI system? What's the retention policy? Is your data used to train future models (and if so, under what terms)? Who has access to query logs and outputs? What happens to that access if the company is acquired, restructured, or goes public?
What's the vendor's incident history? Not the hypothetical catastrophic future incidents they keep warning you about, but the real ones. Data exposures, model manipulation, prompt injection vulnerabilities, API abuse patterns.
How does the vendor handle vulnerability disclosure? The Mythos announcement claims Anthropic partnered with 40+ companies to patch vulnerabilities before hackers could exploit them. That's a disclosure process. Does your AI vendor have one? Do you know how they'd notify you if a vulnerability in their system created exposure for yours?
And critically: what are the false positive rates, error rates, and accuracy benchmarks for any AI tool being used in a decision-making capacity at your organization? The absence of that information in a vendor's marketing materials isn't a minor omission. It's the whole question.
The Real AI Risks Are Operational, Not Existential
Emily M. Bender (professor of computational linguistics at the University of Washington and co-author of "The AI Con") describes the doomsday narrative as a misdirection: "They're saying 'look over here', never mind the environmental destruction and the labour exploitation and all these systems we're destroying in society."
For a mid-sized business, AI tools present real business risk that has nothing to do with extinction scenarios. AI tools embedded in your workflows are touching customer data, generating outputs that inform business decisions, and in some cases interfacing directly with your systems. Third-party AI risk lives in your vendor stack whether you've formally assessed it or not.
The good news is that none of this is ungovernable. Vallor's closing argument in the BBC piece applies just as well to organizational risk management as it does to international AI policy: "Nothing about them is ungovernable. Unless we choose not to govern them."
A structured vendor risk assessment for AI tools gives you the framework to evaluate them the same way you'd evaluate any third party with access to sensitive systems or data. Not with awe, and not with panic. With the same methodical questions that surface real risk and give you something actionable to work with.
The companies selling you AI tools would prefer you spend your energy worrying about robot uprisings. Your job is to worry about access controls, data handling, and contractual obligations instead.
Those are problems you can actually solve.
Nearest Solutions offers quarterly vendor cybersecurity reviews that assess your vendors' security posture on an ongoing basis, including AI tool vendors. Learn more about our vendor review process.