OVER the past 12 months, the capabilities of frontier AI models have remained a central topic of debate. Recent performance demonstrations across a variety of technical fields (maths and bio-genetics in particular) show that scepticism around raw capability at least is a dated position. However, there are still question marks around financial viability, regulation of broad-based societal risks, and impact on workers and labour more generally. The latter two deserve more scrutiny.

The first concern, around broad-based risks, has gained quite a bit of attention with a spate of resignations from top AI companies, with departing individuals claiming that AI poses a serious risk to society and carries a non-zero chance of triggering human extinction.

What is the pathway of such a catastrophe? The most common position at this point is that, in the absence of clear ethical guardrails, AI tools can be used to create biological weapons of the sort that could trigger large-scale human loss. Others think that, in the wrong/rival hands, AI could be used to trigger weapons of mass destruction.

These fears appear to be fairly fantastical, and the timing with which they’re being made public — just when the frontier models are poised for initial public offerings — suggests they’re part of a Silicon Valley hype cycle.

However, if these claims about capabilities and the risks they could carry are even remotely true, they pose a much simpler question. If a technology is truly this capable, then what is it even doing in private........

© Dawn