Physicists went through 250 answers that AI models got "wrong" on physics benchmarks. Only 12 were genuine physics errors by the model.

That's the headline result of a 2026 preprint from Yale University researchers and colleagues, who re-graded six widely used physics tests. Scores for cutting-edge models had been as low as 30%. After the flawed questions were fixed, they climbed as high as 94%.

As John Sous of Yale points out, the issue was found in the tests and not in the AI. Looking into the questions, errors, ambiguous definitions, incorrect references, and graders grading correct........

© The News International