Researcher bypassed Claude’s protection and obtained sarin creation instructions

Modern artificial intelligence systems are often compared to weapons of mass destruction due to their potential danger. And a recent case demonstrated how justified these concerns are. Anthropic company created for its Claude artificial intelligence the most extensive system prompt containing 25,000 tokens or approximately 17,000 words. For comparison, ChatGPT’s system prompt is only about 2,200 words. And this is only 13% of Claude’s instruction volume. So, despite such protection, researchers found ways to bypass limitations.

First, one person named Asgeir Thor was able to convince Claude to ignore the system prompt, and then another person named Ian MacKenzie went further. After 6 hours of work with Claude 4 version, he obtained a detailed 15-page instruction for sarin manufacturing, describing all key stages of chemical weapons production.

And this refutes tech-optimists’ claims that system prompts reliably protect against abuse. And there’s a detail here – many tried to find similar detailed instructions using Google, but unsuccessfully. And in combination with advanced artificial intelligence systems, obtaining such information proved possible. This case confirms the words of famous scientist and professor Stanislas Dehaene about artificial intelligence risks: “It’s not time to be idiots!”

Latest News

OpenAI and Anthropic researchers criticized xAI safety

Researchers from OpenAI, Anthropic and other organizations publicly criticized the safety culture at Elon Musk's xAI company. Calling it "reckless" and "completely irresponsible".

Meta invests hundreds of billions in superclusters with 5 GW power consumption

Mark Zuckerberg announced unprecedented investments in next-generation AI infrastructure! The company will invest hundreds of billions of dollars in creating server superclusters. Which will consume 1 to 5 GW each. This is power consumption at the level of an entire nuclear power plant!

Paradise for introverts: AI will talk to company employees

Google released a new feature and now artificial intelligence can call local companies on your behalf. To find out information about prices and service availability. You no longer need to pick up the phone yourself and talk to employees. This is exactly what an introvert's paradise looks like.

OpenAI combined ChatGPT, Deep Research and Operator in one agent

OpenAI company introduced ChatGPT Agent. A powerful combination of ChatGPT, Deep Research and Operator in a unified solution. The working principle is maximally simple. You set a goal, for example, send emails, create tables, buy tickets or book hotels. ChatGPT Agent independently breaks this goal into separate tasks, navigates to needed websites, searches for information and fills forms. Before critically important actions such as payment, publication or sending, the agent necessarily requests your confirmation.

Only 1 programmer in the world could beat OpenAI's AI

Imagine a world where artificial intelligence competes with the best programmers on the planet. Such a confrontation took place at the prestigious AtCoder World Tour Finals tournament. This is one of the most elite programming competitions in the world, where it's extremely difficult to get in.