The AI mirror metaphor [6] foregrounds the threat of large language models (LLMs) absorbing human error and bias from their training sets and rehearsing them in their output. Key to the concern is that inherent limitations prevent LLMs from critically engaging with their training sets: in short, LLMs do not think [3]. In this talk I raise a complementary worry: that humans could be inherently vulnerable to error and bias when engaging with LLM output, stemming from unmet presuppositions.
I make the case for this as a function of two premises. One premise is that unlike LLMs, humans do think, but remain subject to procedural presuppositions. In the domain of joint language use, a common area for human-LLM collaboration, reasoning-based theories (e.g. [1],[2]) presuppose collaborators who both reason. Competing memory-based theories (e.g. [4], [5]) presuppose shared processing between collaborators who both process language. The other premise is as above: LLMs do not think. Presuppositions of reasoning or processing are unmet by LLMs, who do not reason or co-process, while their observable output appears no different.
The worry I extract from these premises is conditional: given the unmet presuppositions, the success of human-LLM collaboration offers uncomfortable options for human competence. One moderate option is that joint language use and similar domains are less rule-following than major theories suggest and allow extensive omissions, placing humans closer to LLMs. A radical alternative is that in absence of reasoning and processing to support engagement with LLM output, humans implicitly supply ex post facto predictive ‘hallucinations’ of LLM reasoning and processing, which never took place, for their own use. The case for such a mechanism is not exotic [5], and its implication could be that humans implicitly give LLM content background reasoning to suit their own biases, as a mirror for the AI mirror themselves.
REFERENCES
- Clark, H. H. (1996) Using language. Cambridge University Press.
- Grice, P. (1975). Logic and conversation. In Cole, P.; Morgan, J. (eds.), Syntax and semantics. Vol. 3: Speech acts. New York: Academic Press. pp. 41–58.
- Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024). Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517-540.
- Pickering, M. J., & Garrod, S. (2004) Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(2), 169–190.
- Pickering, M. J., & Garrod, S. (2013) An integrated theory of language production and comprehension. Behavioral and Brain Sciences, 36(4), 329–347.
- Vallor, S. (2022) The AI mirror: Reclaiming our humanity in an age of machine thinking. Oxford University Press.
Author: Eugene Philalithis