07-22-2026IMPACT COUNCIL

Is your AI as good as it says it is?

AI can look capable in controlled conditions then behave very differently when it meets the messiness of the real world.

[Photo: Getty Images]

The Fast Company Impact Council is an invitation-only membership community of top leaders and experts who pay dues for access to peer learning, thought leadership, and more.

AI has become very good at passing the tests we set for it.Stanford University’s 2026 AI Index Report captures the problem neatly. While AI models can master abstract logic, they often struggles with basic spatial reasoning tasks. For instance, a leading AI model could win gold at the International Mathematical Olympiad, yet it could correctly read an analogue clock only half the time.

That is the paradox of AI. Exceptional in one domain. Unreliable in another.

That unevenness matters because AI is rarely marketed as conditional, only as capable.

We have seen the same pattern play out on some of the biggest stages in tech. For instance, Tesla’s Optimus robots were presented as a glimpse of autonomous robotics, but after their appearance at a 2024 event it was reported they relied on human intervention for some capabilities. Meta’s AI glasses failed twice during live demos, with the company later pointing to technical issues.

The details differ, but the pattern is consistent: AI can look capable in controlled conditions and then behave very differently when it meets the messiness of the real world.

The same gap is showing up across industries. MIT’s Project NANDA Gen AI study found that 95% of organizations are getting zero return, with most systems stuck without measurable P&L impact.

The question worth asking is........

© Fast Company