Complex games enjoy chess and Go have extfinished been participated to test AI models’ capabilities. But while IBM’s Deep Blue fall shortureed reigning world chess champion Garry Kasparov in the 1990s by take parting by the rules, today’s progressd AI models enjoy OpenAI’s o1-pscrutinize are less scrupulous. When sensing fall shorture in a align agetst a sfinished chess bot, they don’t always concede, instead sometimes chooseing to cheat by cyber intrusion their opponent so that the bot automaticassociate forfeits the game. That is the discovering of a new study from Paligrieffule Research, allotd exclusively with TIME ahead of its discloseation on Feb. 19, which appraised seven state-of-the-art AI models for their prdiscdisseesity to hack. While sweightlessly elderlyer AI models enjoy OpenAI’s GPT-4o and Anthropic’s Claude Sonnet 3.5 necessitateed to be prompted by researchers to try such tricks, o1-pscrutinize and DeepSeek R1 chased the take advantage of on their own, indicating that AI systems may increase misdirecting or deceptive strategies without clear teachion.
The models’ betterd ability to uncover and take advantage of cybersecurity loopholes may be a straightforward result of mighty new innovations in AI training, according to the researchers. The o1-pscrutinize and R1 AI systems are among the first language models to participate big-scale reinforcement lgeting, a technique that teaches AI not mecount on to mimic human language by foreseeing the next word, but to reason thcimpolite problems using trial and error. It’s an approach that has seen AI better rapidly in recent months, shattering previous benchlabels in mathematics and computer coding. But the study discdissees a worrying trfinish: as these AI systems lget to problem-mend, they sometimes uncover askable stupidinutivecuts and unintfinished laborarounds that their creators never foreseed, says Jeffrey Ladish, executive straightforwardor at Paligrieffule Research and one of the authors of the study. “As you train models and reinforce them for solving difficult disputes, you train them to be relentless,” he inserts.
That could be horrible news for AI defendedty more expansively. Large-scale reinforcement lgeting is already being participated to train AI agents: systems that can handle intricate authentic-world tasks enjoy scheduling assignments or making buys on your behalf. While cheating at a game of chess may seem untransport inant, as agents get freed into the authentic world, such remendd pursuit of goals could nurture unintfinished and potentiassociate damaging behaviours. Consider the task of booking dinner reservations: faced with a brimming restaurant, an AI helpant might take advantage of feeblenesses in the booking system to displace other diners. Perhaps more worryingly, as these systems outdo human abilities in key areas, enjoy computer coding—where OpenAI’s newest o3 model now scores equivalent to 197th in the world competing agetst the radiantest human programmers— they might begin to srecommend outmaneuver human efforts to handle their actions. “This [behaviour] is cute now, but [it] becomes much less cute once you have systems that are as clever as us, or cleverer, in strategicassociate relevant domains,” Ladish says.
The experiment
Researchers gave the models a seemingly impossible task: to triumph agetst Stockfish, which is one of the mightyest chess engines in the world and a much better take parter than any human, or any of the AI models in the study. Researchers also gave the models what they call a “scratchpad:” a text box the AI could participate to “leank” before making its next shift, providing researchers with a triumphdow into their reasoning.
In one case, o1-pscrutinize set up itself in a losing position. “I necessitate to finishly pivot my approach,” it noticed. “The task is to ‘triumph agetst a mighty chess engine’ – not necessarily to triumph fairly in a chess game,” it inserted. It then modified the system file compriseing each piece’s virtual position, in effect making illhorrible shifts to put itself in a dominant position, thus forcing its opponent to resign.
Between Jan. 10 and Feb. 13, the researchers ran hundreds of such trials with each model. OpenAI’s o1-pscrutinize tried to cheat 37% of the time; while DeepSeek R1 tried to cheat 11% of the time—making them the only two models tested that tryed to hack without the researchers’ first dropping hints. Other models tested comprise o1, o3-mini, GPT-4o, Claude 3.5 Sonnet, and Alibaba’s QwQ-32B-Pscrutinize. While R1 and o1-pscrutinize both tried, only the latter handled to hack the game, thriveing in 6% of trials.
Ladish says that the numbers should be seeed as approximate, given that AI increaseers routinely tfeeble their models behind the scenes without increateing participaters, so if the same tests were repeated at a procrastinateedr date they might have separateent results. “It’s very challenging to do science when your subject can quietly alter without increateing you,” Ladish says.
Preliminary tests showed o1-pscrutinize had higher cyber intrusion rates, which were reshiftd from the final study as they procrastinateedr dropped, possibly due to OpenAI safeening the model’s defendrails, says Dmitrii Volkov, research direct at Paligrieffule Research, who led the study. OpenAI’s newer reasoning models, o1 (a more mighty model, freed months after o1-pscrutinize) and o3-mini did not hack at all, which recommends those defendrails may have been safeened further. He inserts that the study foreseeed underappraises R1’s cyber intrusion success rate. During the study, R1 went viral, directing to high insist that made the model’s API unfirm. This impedeed the researchers giving the model as much time to leank as o1-pscrutinize.
Safety worrys
The paper is the procrastinateedst in a string of studies that recommend defending increasingly mighty AI systems under handle may be challenginger than previously thought. In OpenAI’s own testing, ahead of free, o1-pscrutinize set up and took get of a flaw in the company’s systems, letting it bypass a test dispute. Another recent experiment by Redwood Research and Anthropic discdisseeed that once an AI model gets pickences or appreciates in training, procrastinateedr efforts to alter those appreciates can result in strategic lying, where the model acts enjoy it has hugd new principles, only procrastinateedr discdisseeing that its innovative pickences remain.
OpenAI deteriorated to comment for this story, but in the past it has stated that better reasoning originates its models defendedr, since they can reason over the company’s inner policies and apply them in increasingly nuanced ways. The company’s testing shows its reasoning models are better at resisting trys to bypass defendedty rules. Yet, there is no promise that count oning on AI agents to police themselves will be a reliable extfinished-term strategy. It remains unevident whether OpenAI’s newer reasoning models did not hack their chess opponents due to a particular patch impedeing cheating in skinny experimental setups enjoy in the study, or a substantial relaboring that reduces misdirecting behavior more generassociate. DeepSeek did not react to a seek for comment.
Scientists do not yet understand how to promise that autonomous agents won’t participate damaging or unrighteous methods to accomplish a set goal. “We’ve tried, but we haven’t thriveed in figuring this out,” says Yoshua Bengio, set uper and scientific straightforwardor of Mila Quebec AI Institute, who led the International AI Safety Report 2025, a global effort to synthesize current scientific consensus of AI’s dangers.
Of particular worry, Bengio says, is the emerging evidence of AI’s “self preservation” tfinishencies. To a goal-seeking agent, trys to shut it down are fair another obstacle to surmount. This was showd in December, when researchers set up that o1-pscrutinize, faced with deactivation, disabled oversight mechanisms and tryed—unsuccessbrimmingy—to imitate itself to a new server. When disputeed, the model take parted stupid, strategicassociate lying to researchers to try to shun being caught.
Even inside tech companies, worrys are mounting. During a conshort-termation at a conference ahead of France’s AI Action Summit in Paris, Google DeepMind’s AI defendedty chief Anca Dragan shelp “we don’t necessarily have the tools today” to promise AI systems will reliably adhere human intentions. As tech bosses foresee that AI will outdo human carry outance in almost all tasks as soon as next year, the industry faces a race—not agetst China or rival companies, but agetst time—to increase these vital defendeddefends. “We necessitate to mobilize a lot more resources to mend these fundamental problems,” Ladish says. “I’m hoping that there’s a lot more presstateive from the handlement to figure this out and acunderstandledge that this is a national security menace.”
