I took an algorithm to court in Sweden. The algorithm won
We like to imagine that injustice announces itself loudly. That when something goes wrong in the public system, alarms go off and someone takes responsibility or is held accountable if they do not. But in 2020 in Gothenburg, injustice arrived quietly, disguised as efficiency.
For the first time, the city used an algorithm to allocate places in its schools. After all, working out geographical catchment areas and admissions is an administrative headache for any municipality. What better than a machine to optimise distances, preferences and capacity? The system was designed to serve public efficiency: framed as neutral, streamlined and objective.
But something went terribly wrong. Hundreds of children were allocated places in schools miles from their homes – across rivers and fjords, over major highways, in neighbourhoods they had never visited and had no connection to. Parents stared at the decisions in disbelief. Had anyone checked whether a 13-year-old could reasonably walk that route in winter? What rationale guided these decisions? Were their stated preferences simply ignored? No one in the schools administration seemed able – or willing – to explain what had happened or to address the errors.
I watched this unfold as a researcher in technology and a former lawyer, but also as a mother. My then 12-year-old son was among the children affected by the algorithm. Our frustration grew with the schools administration’s lack of response. Calmly, they told us we could appeal if we had an issue with our placement – as if it were a matter of taste. As if the problem was due to individual dissatisfaction rather than systemic malfunction. Around kitchen tables across the city, the same confusion and anger simmered. Something was off, and the severity of the problem was becoming increasingly clear.
It was nearly a year before city auditors confirmed what many of us had suspected; the algorithm had been........
