AI Tools are proliferating at an astounding rate. The first general AI chatbot, CHAT-GPT was released on March 30, 2022, and since then AIs have been rocking the world. They respond to queries with very articulate, human-sounding responses. They can write blog posts, essays, technical reports, advertising jingles, summary research reports and literary reviews and even produce half-good poems and jokes. But here’s the unsettling question: do they actually understand what they are saying?
What are AI programs?
CHAT-GPT and most of the AI programs that are now flooding the world, are Large Language Models (LLMs). The models are trained on huge sets of human language sources, including volumes of material scooped from the internet. They are programmed with sophisticated algorithms for classifying sequences of words (many trillions of them) and the probabilities of specific word sequences. These efforts require enormous computing resources and consume prodigious quantities of energy. The result? An amazing, almost magical, capability to assemble lucid and stunningly coherent written responses to questions and task prompts from humans. The quality of these responses to most inquiries do seem to suggest that the programs really understand language and the concepts that they discuss. Do they? How does what LLMs do compare to what humans do?
Training on Data vs. Learning from Meaning
Having to deal with cancer treatments for the past two years (see: My Cancer) has given me time to do more reading that usual. I’ve enjoyed regularly reading articles in dozens of trusted sources dealing with science, philosophy, culture, politics, religion, medicine and current events. I’ve also been able to read quite a few newly published books dealing with physics, biology, neuropsychology, technology, history, philosophy, theology and climate.
The depth and range of materials I have been reading reminds me of the references to the large training data sets that are the basis of the AI models driving the current revolution. How different am I from an LLM AI? Am I just absorbing and categorizing a large data set in order to be able to articulate and repeat sequences of words that seem to me to make sense of the mash of words I have been reading? Do the new sequences match with or update my prior data training and the provisional (or perhaps hard-wired) sequencing of words through which I have been describing the world and my life?
The simple answer is NO. What I and other humans do is quite different from what AI does. We categorize information on the basis of meaning. We understand what words mean. Our cognition operates by extracting meaning. AI works by extracting probabilities on sequences.
Reading as a human
My encounter with the material listed above that I have been “training on” comes with a level of understanding (not always complete or perfect) about the language of the written material and what it means. This is a far more sophisticated engagement with language data than what the AIs are doing. They look at the probabilities of certain words following other certain words without any process by which they might understand the linguistic meaning of what they are interrogating. I look at the references and the meaning of the words and frame them within a context of prior understanding.
Admittedly, it is conceivable that what my brain does at the neural level is not unlike what the AI does – changing neural firing probabilities based on linguistic associations. The neuron, like the AI, however, also does not “understand.” But whatever the neurons do, our conscious mind assembles the patterns into an understanding of the language element’s meaning. AIs do not do this.
I’m not going to speculate about whether AIs are just a hop, skip and a jump from a similar understanding of language elements, but the current generation of AI will simply never get there. They do not understand what they are reading. Humans do. Mostly.
The Art of Reading
Understanding written language involves extracting, with whatever exactitude and depth we choose, the meaning that the author is trying to convey. What was the author thinking when he or she wrote those words? Is my understanding clear and complete? Quite often the author will have a technical or experiential depth that I do not, but a good author will use language that conveys that meaning in a way most readers can understand. That is a hugely valuable asset of language, when well used by an author and well attended to by a reader. Careful writing and reading gives us access to a huge range of knowledge not otherwise available. It is a fabulous shortcut to learning – you do not have to experience everything for yourself.
There are times when I’m pretty sure I get the author’s point, so I can skip over some of the detail he or she is offering. A high-quality skim can yield a good overall understanding without delving down “into the weeds.” There is a risk of missing something important, but the saving of time and attention can be worth the risk, depending on the reader’s constraints and interests. Authors can also be tedious, confused, repetitive and self-congratulatory – skipping through or over that material can improve the reading experience without a loss of understanding. Sometimes an author is just flat out wrong: drawing the wrong conclusions, providing only rhetorical commentary, being obviously biased, or even making things up — hallucinating just like an AI!
When I am reacting negatively to someone’s ideas, however, it does raise a caution flag. The author might be trying to say something valid. I may be biased myself. The difference between speculation and “hallucination” can be subtle, a mere choice of words. There is even a chance, however remote it may be, that the material contains useful or insightful nuggets that I should pay attention to.
I also use helpful shortcuts for framing the information and insights of an author. Titles, chapter and section headings, opening and closing paragraphs, first and last sentences, quotes and notes often contain particularly useful points of reference that can guide and enrich the learning experience.
There are also times when we can completely miss the message. This can sometimes, but not always, be attributed to bad writing or, on the flip side, to lazy reading. Sometimes, we just may not have the patience to wade through an author’s detailed exposition, or the deep background needed to understand his or her references. This can be particularly challenging in the fields of philosophy, theology and technical science literature. Admittedly, some of the writing in these areas is incomprehensible at times, but important insights may also be conveyed, sometimes constructed on an extensive framework of prior works or teachings. There can also be deeply subtle and valuable insights hidden in very difficult, obscure texts. As in a mystical koan, true understanding may be layered in meanings that extend beyond the words on the page. In these cases, “literal” readers will never access the meaning being conveyed.
Also, no matter how deeply trained, AIs cannot do any of this.
So what is the real value of AI – at this point?
I have a deep respect for the incredible usefulness of AIs. I have experimented with AI, primarily on Perplexity.ai and on the unavoidable google AI search assistant. I have been astonished at the speed, accuracy and usefulness of the responses. AI is by far the best general research tool ever invented, at least on subjects that are broadly represented in the public literature. I trust it, and use it, to expand my research reach, speed, and depth. In fact I used it as a tool in helping me edit this blog post. AI can prepare excellent and accurate summaries, reviews and critiques of any written materials, including my own writing. It provides excellent responses to my requests for topical overviews and technical explanations. It can quickly and accurately find citations supporting or refuting particular claims or even quotes. AI is a marvelous and quickly evolving tool that has forever altered the research environment for academics, researchers or casual writers like me. I am adamant, however, that I will never use an AI to write material and claim it as my own. But for editing and support – absolutely.
The metaphor that works for me to explain what AI is and does is this: AIs are tools that can interrogate and then recapitulate quickly and in very useful forms, the entirety of human knowledge. It is a wonderful thing. Research will never be the same. But AIs do not understand the material they are trained on and cannot be trusted to offer original insights. Yet.
