Large Language Bureaucracy
AI cannot, as hummans cannot, make infallible decisions when there are "multiple incommensurable demands".
Dense, long-form academic socio organisational overview of AI and the promise of efficient and rational bureaucracy. The article does not offer answers but advocates for preparedness to see the necessity to study the outcomes as they arise. AI cannot, as hummans cannot, make infallible decisions when there are "multiple incommensurable demands".
Some snippets:
If we are unimpressed by stories about paperclip maximizers remaking the galaxy, omniscient bureaucracies of terror or wonder, markets that suddenly become self-aware, and the like, it is not because we think they are too weird. Rather, they are not nearly weird enough, and miss how much of the weirdness is already here. The possible futures we face are much messier and more varied than stark visions of omnipotent AGI, just as our immediate past was. They will be shaped by the collision between imperfect and highly complex technologies and imperfect and highly complex human social systems (Matias, 2023; Nelson, forthcoming). It is impossible to predict the consequences, but we can map, study, and think about them as they are happening.
Rather than expecting AGI to resolve perennial problems of human social organization, we should treat AI as a new social technology which will alleviate some problems, exacerbate others, and create new ones, just as other social technologies have done in the past. That, in turn, suggests the urgency of cooperation between social and computer scientists to figure out its social consequences, and broader social and political coordination too, of the kind that happened in previous stages of the Long Industrial Revolution.
Yet such small-bore representations are hopelessly inadequate for modern human societies, which require impersonal social and informational technologies that can summarize social relations at very large scale. Rather than tracking a few individuals in a close-knit hunter-gatherer community where everyone knows everyone well, or even a village or town, we need to manage interactions that may involve millions—even billions—of people at once. Building and improving the means to do this has involved the development of institutions such as markets, bureaucracies, and even democracy that can handle relatively impersonal relationships at scale, using coarse-grainings that make these relationships comprehensible.
This vast increase in the social capacity for complex information processing has enabled collective cognition and problem solving at a historically unprecedented scale. Social technologies like markets, bureaucracy, and democracy allow human beings to become what economic historian Brad DeLong (2026) calls an “anthology intelligence,” capable of deploying accumulated cultural knowledge in a coordinated way towards large-scale ends. That is their positive aspect. In their negative, these systems regularly appear monstrous to those who find themselves at the wrong end of the power relations they create. Markets, bureaucracies, and even democracies have furthermore devoured older and more intimate forms of social organization, replacing them with vast systems that are regularly indifferent, and sometimes inimical, to the particular fates and desires of individuals and groups.
This is true across the various forms of AI that are deployed today. When social media platforms guess which post to serve or movie to recommend, they match particularized ‘embeddings’—coarse-grainings of information about the particular users and the universe of content—to arrive at their predictions. The LLMs that we emphasize are no more than coarse-grainings of the vast corpora of textual information that they have been trained on, post-processed to seem more natural in their interactions with humans and carry out more complex tasks. They are also no less. It is astonishing that we now have manipulable representations of entire bodies of human culture which can be set to work via an ordinary language interface to produce new outputs. These technologies are a new stage in the trajectory of institutional and organizational development that has run through modernity and the Long Industrial Revolution, giving new ways of managing complexity while creating their own complexities too.
Member discussion