Regulating tools, ignoring power
The concentration of compute and the politics of "trustworthy AI"
This event is part of a demonstration of what Lera offers for running knowledge events. Its content is written with care and draws on real sources, but the conference has not taken place, and apart from the convener its participants are fictional.
In 2019 the European Commission's High-Level Expert Group published its "Ethics Guidelines for Trustworthy AI": seven requirements, from human agency and oversight to transparency and accountability. From then on, "trustworthy" became the brand of European AI policy, and the AI Act was built on the same ground.
Matteo Ferrandi asks: trust in whom? The guidelines, and the Act after them, regulate "systems", that is, products placed on a market. But the decisive fact of this period is not the product. It is the concentration of the means of production: compute, data and expertise in the hands of a small number of companies, most of them American. In 2021 Meredith Whittaker showed that even academic research in AI has come to depend on the resources of these companies, and she called this condition "capture". Kate Crawford describes AI as an extractive industry that feeds on mines, labour and data.
The consequence is that regulation which certifies products may legitimise that very concentration. The cost of compliance is bearable for the giants and heavy for small, public and non-profit actors. "Trustworthy AI" may become a seal that only the powerful can afford.
This has direct consequences for knowledge institutions. Universities, libraries and publishers increasingly rent models from these same few companies, and the infrastructure of reading and writing turns into a monthly subscription. Shoshana Zuboff has written of "behavioural surplus": what users do becomes the company's raw material. In a knowledge institution, that raw material is the work of readers and writers.
Ferrandi proposes a political economy of AI rather than a product ethics: public compute as a commons; open models governed by public institutions; procurement rules that favour interoperable, replaceable services; and the principle that no knowledge institution should depend on a provider it cannot leave. Regulating risk without regulating power treats the symptom, not the disease.
Questions for discussion
- Could a "right to leave the provider" be made binding for libraries and universities?
- Who should run public compute so that it becomes a tool of neither the state nor the market?
- If trust cannot be placed in a product, in which institution should it be placed?
References
High-Level Expert Group on Artificial Intelligence. 2019. Ethics Guidelines for Trustworthy AI. Brussels: European Commission.
Whittaker, Meredith. 2021. "The Steep Cost of Capture." Interactions 28 (6): 50–55.
Crawford, Kate. 2021. Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. New Haven: Yale University Press.
Zuboff, Shoshana. 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. New York: PublicAffairs.