https://sci-fi-ua.netlify.app/" rel="noopener noreferrer" target="_blank">I write about Ukrainian science fiction as intellectual history and about the philosophy of technology. For my first post here I would like to share a question that concerns everyone who teaches, assesses or learns: when a machine gives a correct answer, has it understood anything?

In 1980 John Searle imagined a man locked in a room, following a rule book to answer Chinese questions he cannot read. People outside receive perfect answers, yet nobody inside understands Chinese. For four decades the Chinese Room served as the standard argument against machine understanding. Large language models have made the thought experiment an everyday reality: fluent, relevant, often correct answers produced by manipulating symbols.

In my preprint AI: The Illusion of Understanding I look at the problem from the other side. Human understanding is less stable than Searle assumed. Students learn formulas by heart and reproduce them correctly... sometimes. Experts rely on habits they cannot explain. Readers are convinced by texts that only look coherent. Drawing on Baudrillard’s idea of the simulacrum, I argue that the line between “real” and “simulated” understanding is harder to draw than the debate supposes. A pragmatic question may serve us better: what do we expect an understanding agent to be able to do, and how do we test for it?

For education the stakes are concrete. If an essay, an exam answer or a lab report can be produced without understanding, then our assessments measure output and only assume comprehension. This was true before AI arrived; the machines have simply made it visible.

The full text is open access on Zenodo.

I would be glad to hear from teachers and researchers here: how do you tell understanding from skilled imitation in your own students’ work, and has AI changed the way you check?

Does AI Understand?