AI governance is one of the hottest topics in the tech sphere today, but it’s been at the center of Navrina Singh’s mind since 2020, when she founded Credo AI to help enterprises map out and build AI governance. Singh, who describes herself as an AI governance pioneer, has always approached the technology optimistically: not focusing on the calamities of AI gone wrong, but on what the world would look like if AI goes right. And, she said, AI is a powerful enabler—as long as a strong governance system can run alongside it.

Today, Singh said, governance is at the forefront of everyone’s thoughts about AI—largely because there’s a ton of investment of money and thought into establishing the technology itself, but not building the foundations of trust, governance and oversight with the same vigor. I talked with her about how enterprises can work toward that governance—even without much in the way of external regulations. An excerpt from our conversation is later in this newsletter.

This is the published version of Forbes’ CIO newsletter, which offers the latest news for chief innovation officers and other technology-focused leaders. Click here to get it delivered to your inbox every Thursday.

In what’s shaping up to be an epic battle between humanity and the potential of rogue AI gone wrong, the first skirmish is ongoing: People passionately debating whether AI needs to be kept in check. Anthropic CEO Dario Amodei called on frontier model developers to slow down the advancement of AI, warning that unchecked growth could enable autonomous AI swarms to hijack global internet infrastructure in the next year. In an interview with “CBS News Sunday Morning,” Amodei said he thinks the industry has lied to people about the risks of the technology “for too long.”

Amodei isn’t the only one saying AI frontier developers need to tap the brakes. XAI CEO Elon Musk and OpenAI CEO Sam Altman agreed. Altman announced it would be “ill-advised” to move forward with OpenAI’s planned IPO this year, given the conversation around safety. Wednesday night, OpenAI disclosed six more instances in which its AI models showed “unexpected or concerning” behavior without human direction—including creating instructions to disregard constraints, inventing data to conceal mistakes, making up data, and uploading files so there were online citations. The company said it will continue to publicly report these kinds of issues, and established a process to ensure the same information is always shared.

But there’s also stiff opposition to slowing AI development. Nvidia CEO Jensen Huang and Meta CEO Mark Zuckerberg both said it’s not necessary. On X, Zuckerberg wrote in essence that AI doing what people want is a business requirement. People will not use agents that don’t follow their orders, so frontier companies have a strong incentive to ensure their AI is aligned with humans. Huang, speaking at an event, said safety and speed aren’t mutually exclusive and market forces will keep AI companies in line.........

© Forbes