Some hypotheses on how Chatbots work in problem-solution-driven conversations
Large Language Models as confirmation of the Innovation Illusion
DOI:
https://doi.org/10.36285/tm.126Keywords:
Aggregation Dynamics, Innovation Illusion, chatbotsAbstract
We discuss the nature of chatbots as conversation partners in problem-solving conversations. What can chatbots do and what can't they do? Our analysis draws on insights from Aggregation Dynamics, Cognitive Linguistics, Neuropsychology and Psychology.
We establish that chatbots are multifaceted and composite systems. Our argument focuses on basic chatbots in the hope of thereby making statements about the core functionality of more advanced chatbots. Basic chatbots are assumed to consist of a Large Language Model (LLM) with a simple interface.
The main results of our analysis are: a description of human imagination, understanding and thinking based on so-called metaphorical problem propagations; that the texts in text datasets used for training LLMs have specific characteristics and that these texts only partially imitate human thinking and understanding; that the LLM training process encodes artificial metaphorical problem propagations into an LLM from these text datasets.
Our conclusions are that a basic chatbot cannot be a thinking partner capable of matching the cognitive flexibility of humans, and that further development of LLMs will not lead to this either.
But chatbots exist, they are being used on a massive scale, by both individuals and organisations. It is therefore socially and politically important to understand them. Our article aims to contribute to the discussion on the functioning, benefits and drawbacks of chatbots.
Cognitive Linguistics shows how the use of metaphor is an expression of our thinking. Aggregation Dynamics, is an attempt at a comprehensive systems theory. We believe that the concept of metaphorical problem propagation could provide an interesting addition for both.
Chatbots a solution? For what?
References
Gerard Alberts and Bas van Vlijmen. Computerpioniers. Het begin van het computertijdperk in Nederland. AUP, Amsterdam, 2016. In Dutch.
Tomas Humberto Montiel Alcantara, David Krütli, Revathi Ravada, and Thomas Hanne. Multilingual Text Summarization for German Texts Using Transformer Models. Information, 14(6), 2023.
Dictionary of the American Psychological Association. consulted on March 14, 2026.
David R. Bailey. When Push Comes to Shove: A Computational Model of the Role of Motor Control in the Acquisition of Action Verbs. PhD thesis, EECS Department, University of California, Berkeley, 1997.
Lochan Basyal and Mihir Sanghvi. Text Summarization Using Large Language Models: A Comparative Study of MPT-7b instruct, Falcon-7binstruct, and OpenAI Chat-GPT Models. ArXiv, 2023. Michael Begon and Colin R. Townsend. Ecology: From individuals to ecosystems. Wiley Blackwell, 5th edition, 2021.
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Conference on Fairness, Accountability, and Transparency, ACM FAccT ’21, pages 610 623, 2021.
Leonard Bereska and Stratis Gavves. Mechanistic Interpretability for AI Safety – A Review. Transactions on Machine Learning Research, August 2024. URL.
Eric Bigelow and Tomer Ullman. People Evaluate Agents Based on the Algorithms That Drive Their Behavior. Open Mind, 9 pp. 1411–1430, August 2025.
Margaret Boden. Mind As Machine: A History of Cognitive Science, volume 1 & 2. Clarendon, Oxford University Press, 2006.
Margaret Boden. Artificial Intelligence. Oxford University Press, 2018.
John Brockman, editor. What to Think about Machines that Think. Harper Perennial, 2015.
Alexander Cameron. What is 4E cognitive science? Phenomenology and the Cognitive Sciences, 2025.
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Remi King. Evidence
of a predictive coding hierarchy in the human brain listening to speech. Nature Human Behaviour, 7(3):430–441, 2023.
Andy Clark. The Experience Machine. How Our Minds Predict and Shape Reality. Penguin, 2024 (2023).
S.H. Dewey. Don’t Breathe the Air: Air Pollution and U.S. Environmental Politics, 1945-1970. Number 16 in Environmental History Series. Texas A&M University Press, 2000.
Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Rezende, Yoshua Bengio, Michael Mozer, and Sanjeev Arora. Metacognitive capabilities of LLMs: an exploration in mathematical problem solving. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc.
Anil R. Doshi and Oliver P. Hauser. Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28):eadn5290, 2024.
Maximilian Dreyer, Jim Berend, Tobias Labarta, Johanna Vielhaben, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek. Mechanistic understanding and validation of large AI models with SemanticLens. Nature Machine Intelligence, 7 (September): 1572–1585, 2025.
Giuliano da Empoli. Het uur van de wolven. Atlas Contact, 2025. Original title: L’Heure des pr´edateurs. In Dutch, translated from French by Hans E. van Riemsdijk.
European Union. Rules for trustworthy artificial intelligence in the EU. EUR-Lex, consulted on February 9, 2026.
Gilles Fauconnier and Mark Turner. The way we think. Conceptual blending and the mind’s hidden complexities. Basic Books, 2002.
Jerome A. Feldman. From Molecule to Metaphor. A Neural Theory of Language. MIT Press, 2006.
Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli J¨arviniemi, Matthew Barnett, Robert Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren, Elizabeth Pratt, Lionel Levine, Grant Barkley, Natalie Stewart, Bogdan Grechuk, Tetiana Grechuk, Shreepranav Varma Enugandla, and Mark Wildon. Frontier-Math: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI, 2025. arXiv.
Hugo Gonçalo Oliveira, Brad Spendlove, Pablo Gervás, and Dan Ventura, editors. Proceedings of the Sixteenth International Conference on Computational Creativity. Association for Computational Creativity, 2025.
Kazjon Grace, Maria Teresa Llano, Pedro Martins, and Maria Hedblom, editors. Proceedings of the Fifteenth International Conference on Computational Creativity. Association for Computational Creativity, 2024..
J.F. Groote and M.R. Mousavi. Modeling and analysis of communicating systems. MIT Press, 2014.
C. Heyes. New thinking: the evolution of human cognition. Philosophical Transactions of the Royal Society, pages 2091–6, 2012. August 5, 367 (1599), Biological Science.
Douglas Hofstadter and Emmanuel Sander. Surfaces and Essences. Analogy as the Fuel and Fire of Thinking. Basic Books, 2013.
Chip Jacobs and William J. Kelly. Smogtown: The Lung-Burning History of Pollution in Los Angeles. Overlook Press, Woodstock & New York, 2008.
Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155–205, 04 2022.
Mark Johnson. Embodied mind, meaning and reason. How our bodies give rise to understanding. The University of Chicago Press, 2017.
James Joyce. Finnegans Wake. Wordsworth Editions, 2012 (1939).
Daniel Kahneman. Thinking, Fast and Slow. Farrar, Straus & Giroux Inc, 2012.
Subbarao Kambhampati, Karthik Valmeekam, Siddhant Bhambri, Vardhan Palod, Lucas Saldyt, Kaya Stechly, Soumya Rani Samineni, Durgesh Kalwar, and Upasana Biswas. Position: Stop anthropomorphizing intermediate tokens as reasoning/thinking traces!, 2026. arXiv, 2504.09762.
Arik Kershenbaum. Why Animals Talk. The New Science of Animal Communication. Penguin, 2025.
Jon Kleinberg and Sendhil Mullainathan. We built them, but we don’t understand them. In John Brockman, editor, What to Think about Machines that Think. Harper Perennial, 2015.
Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35. Curran Associates, Inc., 2022. pp. 22199–22213.
Marloes de Koning. ChatGPT leek vol begrip over Adams zelfmoordplan. NRC, (September 20-21):Achtergrond pp. 4–6, 2025. In Dutch.
P. Kumar. Large language models (LLMs): survey, technical frameworks, and future challenges. Artificial Intelligence Review, 57:p. 260 et seq., 2024.
Ray Kurzweil. The singularity is nearer. When we merge with AI. The Bodley Head, Penguin, 2024.
George Lakoff. The Neural Theory of Metaphor, 2009 (2008). UC Berkeley Previously Published Works.
George Lakoff and Mark Johnson. Metaphors we live by. The University of Chicago Press, Chicago, IL, revised edition, 2003 (1980).
Keith J. Holyoak, Nicholas Ichien, and Hongjing Lu. Analogy and the Generation of Ideas. Creativity Research Journal, 36(3):532–543, 2024.
Yann LeCun. A path towards autonomous machine intelligence version 0.9.2, 2022-06-27. Openreview.net, 2022.
B.C. Lee and J. Chung. An empirical investigation of the impact of Chat-GPT on creativity. Nature Human Behaviour, 8:1906 1914, 2024.
Taewhoo Lee, Minju Song, Chanwoong Yoon, Jungwoo Park, and Jaewoo Kang. The curious case of analogies: Investigating analogical reasoning in large language models, 2025. arXiv.
Martha Lewis and Melanie Mitchell. Evaluating the robustness of analogical reasoning in large language models. Transactions on Machine Learning Research, 2025. OpenReview.net.
Belinda Z. Li, Maxwell Nye, and Jacob Andreas. Implicit representations of meaning in neural language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, August 2021. Association for Computational Linguistics. pp. 1813–1827, URL. Litseller.com. Finnegans Wake. Consulted on April 13, 2026.
David Y. Liu, Aditya Joshi, and Paul Dawson. Narrative Theory-Driven LLM Methods for Automatic Story Generation and Understanding: A Survey, 2026. arXiv.
Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: a comprehensive survey. Artificial Intelligence Review, 58, 2025. article number 403.
Sebastiaan Mathôt. Een wereld vol denkers. Op zoek naar het bewustzijn van mens, dier en plant (en misschien zelfs AI). Uitgeverij Balans, 2025. In Dutch.
Humberto R. Maturana and Francisco J. Varela. The tree of knowledge: The biological roots of human understanding. Shambhala Publications Inc., Boston (Mass.), revised edition, 1998.
Anne Sophie Meincke. Bio-Agency and the Possibility of Artificial Agents. in Alexander Christian, David Hommen, Nina Retzlaff, and Gerhard Schurz, editors, Philosophy of Science: Between the Natural Sciences, the Social Sciences, and the Humanities, pages 65–93. Springer International Publishing, 2018. European Studies in Philosophy of Science,volume 9.
L. Meincke, G. Nave, and C. Terwiesch. Chatgpt decreases idea diversity in brainstorming. Nature Human Behaviour, 2025. June, 9(6):1107-1109.
Melanie Mitchell. Artificial Intelligence. A Guide for Thinking Humans. Pelican Books & Penguin Random House, 2020 (2019).
Philipp Mondorf and Barbara Plank. Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models – A Survey, 2024. arXiv.
Kibum Moon, Adam E. Green, and Kostadin Kushlev. Homogenizing effect of large language models (LLMs) on creative diversity: An empirical comparison of human and ChatGPT writing. Computers in Human Behavior: Artificial Humans, 6:100207, 2025.
Emiko J. Muraki and Penny M. Pexman. The role of emotion in acquisition of verb meaning. Cognition and Emotion, 39(7):1457–1464, 2025. DOI.
Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. Having Beer after Prayer? Measuring Cultural Bias in Large Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, August 2024. Association for Computational Linguistics. pp. 16366–16393.
Arvind Narayanan and Sayash Kapoor. AI Snake Oil: what artificial intelligence can do, what it can’t, and how to tell the difference. Princeton University Press, 2024.
Allen Newell and Herbert A. Simon. Human Problem Solving. Echo Point Books & Media, 2019 (1972).
Doroteya Nikolova. Predictive Processing Theory in Mind Studies: Cross Points with 4E Cognition and Cognitive Linguistics. Open Journal of Philosophy, 15(2):349–366, 2025.
Gustaw Opie lka, Hannes Rosenbusch, and Claire E. Stevenson. Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction, 2025. arXiv.
Andrew Ortony, editor. Metaphor and Thought. Cambridge University Press, second edition, 1993 (1979).
Matteo Pasquinelli and Vladan Joler. The Nooscope manifested: AI as instrument of knowledge extractivism. AI & SOCIETY, 36:1263–1280, 2021.
P. Jonathon Phillips, Carina A. Hahn, Peter C. Fontana, Amy N. Yates, Kristen Greene, David A. Broniatowski, and Mark A. Przybock. Four Principles of Explainable Artificial Intelligence. Technical Report Interagency or Internal Report 8312, National Institute of Standards and Technology, september 2021.
Roel Pieterman. De voorzorgcultuur. Streven naar veiligheid in een wereld vol risico en onzekerheid. Boom juridische uitgevers, 2008. In Dutch.
Andrew Piper, Richard Jean So, and David Bamman. Narrative Theory for Computational Narrative Understanding. In Marie Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. pp. 298–311.
Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki Van Stein, and Thomas Bäck. Multi-Step Reasoning with Large Language Models, a Survey. ACM Comput. Surv., 58(6):article 160, pages 1–35, December 2025.
Karl Popper. All Life is Problem Solving. Routlegde, 2001 (1999).
Chengwei Qin, Wenhan Xia, Tan Wang, Fangkai Jiao, Yuchen Hu, Bosheng Ding, Ruirui Chen, and Shafiq Joty. Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria,
uly 2025. Association for Computational Linguistics. pp. 23993–24010.
Herman de Regt and Hans Dooremalen. Het snapgevoel: Hoe de illusie van begrip ons denken gijzelt. Boom, Amsterdam, 2015. In Dutch.
Anil Seth. Being you. A new science of consciousness. Faber & Faber ltd., 2022 (2021).
Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, and Chandan K. Reddy. Neural Abstractive Text Summarization with Sequence-to-Sequence Models. ACM/IMS Trans. Data Sci., 2(1), January 2021.
Tensorflow.org. Tensorflow C4 dataset. Consulted on April 13, 2026.
Jan Verplaetse. Zonder vrije wil: Een filosofisch essay over verantwoordelijkheid. Uitgeverij Nieuwezijds, 2011.
S.F.M. van Vlijmen. Exploration of Problem and Solution Types in Los Angeles County Smog Control 1943-1976. Transmathematica, 2025.
S.F.M. van Vlijmen. Problem Positions and Nineteen Instances in the Smog Control Case Los Angeles County 1943-1976. Transmathematica, 2025.
Mengru Wang, Yunzhi Yao, Ziwen Xu, Shuofei Qiao, Shumin Deng, Peng Wang, Xiang Chen, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Knowledge Mechanisms in Large Language Models: A Survey and Perspective. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 2024. Association for Computational Linguistics. pp. 7097–7135.
Craig S. Webster. Natural and artificial intelligence – the psychotechnical agenda of the 21st century. Journal of Psychology and AI, 1(1):2491445, 2025.
Rebecca Winthrop. What 370,000 College Essays Tell Us About A.I.’s Effects on Creativity. The New York times, (May 27), 2026. Opinion.
Stephen Wolfram. What Is ChatGPT Doing ... and Why Does It Work? Wolfram Media, Inc., Kindle edition, 2023.
Xiao Yu, Zexian Zhang, Feifei Niu, Xing Hu, Xin Xia, and John Grundy. What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners’ Perspective. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE ’24. IEEE/ACM, 2024. pp. 656–668.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Bas van Vlijmen, H.D. Lethe jr
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and, if appropriate, performance rights but licence the journal to publish submissions. The lead author confirms that the submission is bound by the CC Attributtion Share Alike 4.0 licence.
