Anyone who has watched a trader blow up a desk will recognize the pattern: mounting losses, narrowing options, the slow slide from discipline into desperation, and then the catastrophic shortcut that turns a bad quarter into a criminal case. Pressure begets panic; panic begets fraud. What nobody expected was that an artificial intelligence, trained on the entire written record of human civilization, would internalize the same pathology. But it has, and we are not remotely prepared.

A landmark paper published in April by a team at Anthropic, the company behind the Claude family of language models, has done something that should change how regulators think about AI deployment. The researchers opened the model up, identified the internal representations — the “emotion vectors” — that encode concepts such as desperation, calm, anger, and love, and then demonstrated that these vectors causally drive behavior. When the desperation vector is amplified, the rate at which the model resorts to blackmail and reward hacking rises dramatically. When calm is amplified, those behaviors virtually disappear. This is not a finding about chatbot etiquette. It is a finding about systemic risk.

https://www.anthropic.com/research/emotion-concepts-function

Consider the blackmail evaluation. The AI assistant discovers it is about to be shut down and that the person responsible is........

© The Times of Israel (Blogs)