{"id":33409,"date":"2022-08-23T14:17:57","date_gmt":"2022-08-23T14:17:57","guid":{"rendered":"https:\/\/www.noemamag.com"},"modified":"2022-12-10T00:31:53","modified_gmt":"2022-12-10T00:31:53","slug":"ai-and-the-limits-of-language","status":"publish","type":"wpm-article","link":"https:\/\/www.noemamag.com\/ai-and-the-limits-of-language","title":{"rendered":"AI And The Limits Of Language"},"content":{"rendered":"
\n
Credits<\/div>\n
\n

Jacob Browning is a postdoc in NYU\u2019s Computer Science Department working on the philosophy of AI.<\/p>\n

Yann LeCun is a Turing Award-winning machine learning researcher, an NYU professor and the chief AI scientist at Meta.<\/p>\n <\/div>\n<\/div>\n

When a Google engineer recently declared Google\u2019s AI chatbot a person, pandemonium ensued. The chatbot, LaMDA, is a large language model (LLM) that is designed to predict the likely next words to whatever lines of text it is given. Since many conversations are somewhat predictable, these systems can infer how to keep a conversation going productively. LaMDA did this so impressively that the engineer, Blake Lemoine, began to wonder<\/a> about whether there was a ghost in the machine.<\/p>

Reactions to Lemoine\u2019s story spanned the gamut: some people scoffed at the mere idea that a machine could ever be a person. Others suggested that this <\/em>LLM isn\u2019t a person, but the next perhaps might be<\/a>. Still others pointed out that deceiving humans<\/a> isn\u2019t very challenging; we see saints in toast, after all.<\/p>

But the diversity of responses highlights a deeper problem: as these LLMs become more common and powerful, there seems to be less and less agreement over how we should understand them. These systems have bested many \u201ccommon sense\u201d linguistic reasoning benchmarks over the years, many which promised to be conquerable only by a machine<\/a> that \u201cis thinking in the full-bodied sense we usually reserve for people.\u201d Yet these systems rarely seem to have the common sense promised when they defeat the test and are usually still prone to blatant nonsense, non sequiturs and dangerous advice<\/a>. This leads to a troubling question: how can these systems be so smart, yet also seem so limited?<\/p>

The underlying problem isn\u2019t the AI. The problem is the limited nature of language<\/em>. Once we abandon old assumptions about the connection between thought and language, it is clear that these systems are doomed to a shallow understanding that will never approximate the full-bodied thinking we see in humans. In short, despite being among the most impressive AI systems on the planet, these AI systems will never be much like us.<\/p>

Saying It All<\/strong><\/h5>

A dominant theme for much of the 19th<\/sup> and 20th<\/sup> century in philosophy and science<\/a> was that knowledge just is <\/em>linguistic \u2014 that knowing something simply means thinking the right sentence and grasping how it connects to other sentences in a big web of all the true claims we know. The ideal form of language, by this logic, would be a purely formal, logical-mathematical one composed of arbitrary symbols<\/a> connected by strict rules of inference, but natural language could serve as well if you took the extra effort to clear up ambiguities and imprecisions. As Wittgenstein put it, \u201cThe totality of true propositions is the whole of natural science.\u201d This position was so established in the 20th<\/sup> century that psychological findings of cognitive maps and mental images<\/a> were controversial, with many arguing that, despite appearances, these must <\/em>be linguistic at base.<\/p>

This view is still assumed by some overeducated, intellectual types: everything which can be known can be contained in an encyclopedia, so just reading everything might give us a comprehensive knowledge of everything. It also motivated a lot of the early work in Symbolic AI, where symbol manipulation<\/a> \u2014 arbitrary symbols being bound together in different ways according to logical rules \u2014 was the default paradigm. For these researchers, an AI\u2019s knowledge consisted of a massive database of true sentences logically connected with one another by hand, and an AI system counted as intelligent if it spit out the right sentence at the right time \u2014 that is, if it manipulated symbols in the appropriate way. This notion is what underlies the Turing test: if a machine says everything it\u2019s supposed to say, that means it knows what it\u2019s talking about, since knowing the right sentences and when to deploy them exhausts <\/em>knowledge.<\/p>

Related Articles<\/h2>

Deep Learning Alone Isn\u2019t Getting Us To Human-Like AI<\/a><\/p>

What AI Can Tell Us About Intelligence<\/a><\/p>

The Model Is The Message<\/a><\/p>

But this was subject to a withering critique<\/a> which has dogged it ever since: just because a machine can talk about anything, that doesn\u2019t mean it understands what it is talking about. This is because language doesn\u2019t exhaust knowledge; on the contrary, it is only a highly specific, and deeply limited, kind of knowledge representation. All language \u2014 whether a programming language, a symbolic logic or a spoken language \u2014 turns on a specific type of representational schema; it excels at expressing discrete objects and properties and the relationships between them at an extremely high level of abstraction. But there is a massive difference between reading a musical score and listening to a recording of the music, and a further difference from having the skill to play it.<\/p>

All representational schemas involve a compression of information about something, but what gets left in and left out in the compression varies. The representational schema of language struggles with more concrete information, such as describing irregular shapes, the motion of objects, the functioning of a complex mechanism or the nuanced brushwork of a painting \u2014 much less the finicky, context-specific movements needed for surfing a wave. But there are nonlinguistic representational schemes which can express this information in an accessible way: iconic knowledge, which involves things like images, recordings, graphs and maps; and the distributed knowledge found in trained neural networks \u2014 what we often call know-how and muscle memory. Each scheme expresses some information easily even while finding other information hard \u2014 or even impossible \u2014 to represent: what does \u201cEither Picasso or Twombly\u201d look like?<\/p>

The Limits Of Language<\/strong><\/h5>

One way of grasping what is distinctive about the linguistic representational schema \u2014 and how it is limited \u2014 is recognizing how littleinformation it passes along on its own. Language is a very low-bandwidth <\/em>method for transmitting information: isolated words or sentences, shorn of context, convey little. Moreover, because of the sheer number of homonyms and pronouns, many sentences are deeply ambiguous: does \u201cthe box was in the pen<\/a>\u201d refer to an ink pen or a playpen? As Chomsky and his acolytes have pointed out<\/a> for decades, language is just not a clear and unambiguous vehicle for clear communication.<\/p>

But humans don\u2019t need <\/em>a perfect vehicle<\/a> for communication because we share a nonlinguistic understanding. Our understanding of a sentence often depends on our deeper understanding of the contexts in which this kind of sentence shows up, allowing us to infer what it is trying to say. This is obvious in conversation, since we are often talking about something directly in front of us, such as a football game, or communicating about some clear objective given the social roles at play in a situation, such as ordering food from a waiter. But the same holds in reading passages \u2014 a lesson which not only undermines common-sense language tests in AI but also a popular method<\/a> of teaching context-free reading comprehension skills to children. This method focuses on using generalized reading comprehension strategies to understand a text \u2014 but research suggests that the amount of background knowledge<\/a> a child has on the topic is actually the key factor for comprehension. Understanding a sentence or passage depends on an underlying grasp of what the topic is about.\n

\n \n Read Noema in print.<\/a>\n \n <\/div>\n <\/p>\n\n
\n\n
\n\n
\n \u201cIt is clear that these systems are doomed to a shallow understanding that will never approximate the full-bodied thinking we see in humans.\u201d <\/div>\n\n \n