Risk categories and the missing language
What the EU's Artificial Intelligence Act does not see
This event is part of a demonstration of what Lera offers for running knowledge events. Its content is written with care and draws on real sources, but the conference has not taken place, and apart from the convener its participants are fictional.
Regulation (EU) 2024/1689, known as the Artificial Intelligence Act, entered into force in August 2024 and applies in stages: the prohibitions from February 2025, the obligations for general-purpose models from August 2025, and most of the rules for high-risk systems from August 2026. Its architecture rests on a classification of risk: prohibited practices, high-risk systems, systems with transparency obligations only, and the rest.
Anneliese Brandt reads the Act with a single question: where is language in it? Systems used in education for admission and assessment, in migration and asylum, and in access to essential public services are classed as high-risk in Annex III. In every one of these areas the person's language is decisive: an asylum seeker whose interview is transcribed or translated by a machine, or a student whose essay in a second language is graded by an automated system. Even so, the Act does not recognise differences in a system's performance across languages as a risk in its own right. Article 10 asks for data that are "sufficiently representative", but representativeness is measured against the system's intended purpose, not against the languages of the people it may harm.
The evidence of the gap is not hidden. In 2020 Joshi and colleagues classified the world's languages by their data and tools and showed that most languages have almost nothing. In 2021 Bender, Gebru and colleagues warned that making models larger does not answer the question "whose data?". A system that works well in English or German and fails in Tigrinya or Dari is a high-risk system for a Dari-speaking applicant, however brilliant its average accuracy.
She makes three concrete proposals. First, conformity assessments for high-risk systems should report performance separately for the languages of the people the system decides about. Second, wherever a high-risk system processes a person's speech or writing in a language other than the Union's official languages, that person should have the right to a human interpreter. Third, the European standardisation bodies should treat language coverage as part of a system's accuracy and robustness, not as a side issue at its margins.
Europe has made multilingualism an institutional value for its twenty-four official languages. In the regulation of AI even that value is muted, and the languages of migrants, the people most exposed to high-risk systems, are absent altogether. A law that measures risk but does not see language has left part of the risk outside its field of vision.
Questions for discussion
- Should a performance gap between languages count as discrimination in law?
- Who should pay for testing systems in low-resource languages: the provider, the deployer or the state?
- How can the right to a human interpreter against an automated decision be guaranteed in practice?
References
Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union L, 12 July 2024.
Joshi, Pratik, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. "The State and Fate of Linguistic Diversity and Inclusion in the NLP World." In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 6282–6293.
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623.