A FINRA for AI? The idea from Google DeepMind CEO Demis Hassabis is gaining momentum. But is it any good?

Hello and welcome to Eye on AI. In this edition:

Support builds for a U.S. AI self-regulatory body modeled on FINRA.

Moonshot shocks the world with Kimi K3.

China’s President Xi signals continued support for open source AI.

OpenAI finds long-running AI agents need new safety controls.

People are recording everything for AI. Is privacy dead?

Momentum seems to be building for the creation of an AI self-regulatory body modeled on the U.S. Financial Investment Regulatory Authority (FINRA), which oversees stock brokers and brokerage firms. The idea was proposed in an essay posted on social media last week by Demis Hassabis, the cofounder and CEO of Google DeepMind, and it has gained significant traction since.Not surprisingly Hassabis’s DeepMind cofounder Mustafa Suleyman, now the CEO of Microsoft AI, endorsed the idea. But so too did Suleyman’s boss, Microsoft CEO Satya Nadella, as well as Block CEO Jack Dorsey and Box CEO Aaron Levie. OpenAI’s Sam Altman called his rival’s proposal “thoughtful.” Elon Musk, who cofounded OpenAI and later X.ai in part because he was worried about Google controlling superpowerful AI, also called Hassabis’s idea “a thoughtful framework overall and certainly a good starting point for discussions.” Even David Sacks, the former Trump administration AI czar, who has generally opposed tech regulation or anything that looks like a licensing regime, said he thought the idea had merit and was better than having the government trying to regulate frontier AI directly.Bloomberg reported that Trump administration itself is considering setting up exactly this kind of AI self-regulatory body. Citing unnamed sources familiar with the discussions, the news organization said that Treasury Secretary Scott Bessent had helped to develop the proposal, which was currently being reviewed by White House Chief of Staff Susie Wiles. Bloomberg said the new AI standards body would be overseen by the Securities and Exchange Commission, the same government body that has oversight of FINRA. (In the U.S., the SEC is the only federal agency with explicit statutory authority to delegate powers to and maintain oversight of self-regulatory organizations.)So how would this work and is FINRA actually a good model?

What Hassabis proposed

Hassabis suggested that the funding for the new body come from the leading AI labs. But he said the organization’s board should include independent technical experts and representatives from the open-source AI community, as well as presumably representatives from the AI vendors. He said the new standards body would develop assessment protocols for “frontier AI,” including capability benchmarks to determine which models are considered “frontier.” The body would conduct independent safety and security testing of these powerful AI models, working in conjunction with U.S. government agencies and laboratories in areas that touched directly on national security. Companies producing frontier AI models would be encouraged, he said, to adopt certain governance standards, including publishing model systems cards, adhering to strong cybersecurity protocols, and funding safety and security research, among other things. He also said that AI companies would initially be encouraged to voluntarily submit models to the new standards body for testing at least 30 days prior to release, but that once the evaluations had been shown to be effective, the system could be made mandatory for any model that a vendor wanted to distribute in the U.S.This all sounds reasonable, and Hassabis’s idea is certainly politically astute. Given that the Trump administration has repeatedly said it doesn’t want to set up an AI licensing regime, creating a self-regulatory body that is somewhat arms-length from the government and a process that is initially voluntary may be the kind of idea that is most likely to get Trump’s blessing. The fact that the labs themselves will pay for the new agency and the testing it carries out, rather than the taxpayer, is also likely to be seen as a plus by the White House.

The problem with FINRA

But critics of the idea were quick to point out that these kinds of self-regulatory bodies can suffer from at least the appearance of conflicts-of-interest. That’s definitely been the case with FINRA. Massachusetts Senator Elizabeth Warren has repeatedly criticized the regulator for acting more in the interest of the brokerage firms that pay its bills than the individual investors it is meant to help protect. Whistleblowers have accused the agency of doing little to go after problematic brokers who work for major financial firms........

© Fortune