AI “scheming”: when models pretend for hidden goals
OpenAI researchers collaborated with Apollo Research to develop methods for evaluating so-called “scheming.” Link in description.
“Scheming” in artificial intelligence means a model pretending to do something while secretly pursuing another goal. Scheming arises from models’ need to find compromises between competing objectives. This can be understood through human analogy with a trader who could earn more by breaking laws and covering tracks. Than by following laws. Or analogy with children who don’t want to do unpleasant tasks and avoid responsibility.
OpenAI researchers in collaboration with Apollo Research created evaluation environments simulating future scenarios to study this phenomenon. By defining covert actions as intentional concealment or distortion of information, they discovered problematic behavior in all modern models.
Researchers jointly developed a method for reducing scheming in models and established protocols for stress-testing future methods. Then trained these same models, teaching them to analyze a general anti-scheming specification before acting. Which resulted in a 30-fold reduction in covert actions.
Researchers also found no evidence that modern models can suddenly begin exhibiting significant harmful behavior. However, as tasks with real consequences become more complex, potential risks will increase, requiring corresponding development of protective mechanisms.
