All Quiet on the Knowledge Front. Why Epistemia, the Illusion of Knowledge, is Nothing New

I find myself using generative AI quite a bit these days. However, I am hesitant to call myself an enthusiast. To be a fan implies a kind of uncritical enthusiasm that I just cannot muster, mostly because I am acutely aware of the limits of these instruments.

An LLM—and while there are other technologies that qualify as artificial intelligence, we are all currently fixated on large language models like ChatGPT, Gemini, and Claude—does not possess an understanding of what it is saying.

I have written about this before, but it bears repeating: these models manipulate text in incredible ways, doing things only humans could do before (and, obviously, doing things humans cannot do, like reading a book in seconds). But they don't understand what they are writing—let alone possess a conscience. They are statistical engines, predicting the next likely token in a sequence, not minds contemplating meaning.

I have previously explored these limits from a moral perspective, asking what we owe to—or fear from—a machine without agency. But a recent research paper conducted by the team of Professor Walter Quattrociocchi has convinced me to return to the topic, this time on an epistemic level.

The paper in question is titled The simulation of judgment in LLMs. The researchers set up a fascinating confrontation between human and artificial evaluations of news articles. They benchmarked six different LLMs against expert human ratings (from NewsGuard and Media Bias/Fact Check) and against a group of non-expert human participants.

The goal wasn't just to see if the AI got the right answer regarding a site's reliability, but to understand how it got there. They used a structured framework where both the models and the humans had to select criteria, retrieve content, and produce justifications.

The results were noteworthy. The models’ outputs often aligned with expert ratings—in fact, they were quite good at flagging unreliable sources. However, the way they reached those conclusions was fundamentally different. The study found that LLMs rely heavily on lexical associations and statistical priors rather than the contextual reasoning humans use. In other words, they look for the shape of a reliable text, not the truth of it.

I fully agree with the authors’ conclusion that we need more public awareness of how LLMs think (or, if that verb annoys you as much as it sometimes annoys me, how they elaborate their outputs). We need this awareness not to avoid using them, but to properly integrate them into our epistemic system.

Nevertheless, I have a problem with a single, perhaps marginal, aspect of this paper. It may be marginal to the data, but I suspect it is crucial to Quattrociocchi’s intentions, given his specific focus on the concept of epistemia itself.

I am speaking about the concept of epistemia.

In the paper, epistemia is defined as the tendency to confuse linguistic form with epistemic reliability. It describes a condition where the appearance of coherent and authoritative judgment arises from statistical patterning alone, producing the illusion of knowledge when surface plausibility substitutes for evidence-based reasoning.

Essentially, it is the trap of believing the machine because it sounds smart.

Why does this concept leave me cold? For two reasons: there is little new in it, and I fear there is little useful in it.

To understand why, we need to take a step back.