This morning, I asked a commonly used Large Language Model whether it was biased against Israel. What followed turned into a long conversation in which the LLM at first defended itself but then gave away concessions.

At 6,000 words, that conversation is too long to be printed here, so I offer a summary of the exchange by another equally well-known LLM.

The original conversation can be found on my Facebook page: https://www.facebook.com/StephenNGames/

THE EXCHANGE BEGINS with a provocative but testable proposition: if an AI machine is trained on a vast body of human-produced material, and if that material is disproportionately hostile to Israel, then the AI should itself be susceptible to an anti-Israel tilt—even if later “balance” mechanisms try to correct it. 

The AI’s first instinct is to resist almost every link in that chain. It objects that its training material is not simply “the web”; questions whether the web is in fact predominantly hostile to Israel; points out that Western sources have historically contained considerable pro-Israel material; and stresses that post-training procedures are intended to reduce inherited bias. Its fallback position is therefore: bias is possible, but there is no evidence here for its direction. 

That position then begins to erode.

When you move from the abstract “web” to things that can actually be measured—UN voting, government positions and international opinion—the AI first complains about methodology, but eventually concedes that the diplomatic evidence is very heavily weighted against Israel and that Western governmental support has also weakened. Importantly, it admits that it had described this area as more uncertain than it really was. 

You then broaden the case: the relevant environment is not merely diplomatic. It includes professional journalism, activism, opinion writing and public sentiment. At this point the discussion makes its most important turn. The AI actually checks polling and discovers that the evidence is considerably stronger than it had allowed. It cites international polling showing markedly negative attitudes to Israel across many regions and acknowledges that the mechanism you proposed—a preponderance of critical material exerting a directional pull on model........

© The Times of Israel (Blogs)