Step off giants’ shoulders

Language is central to human experience, to our interaction with the external world. Even when we experience silence, it is through language! Our entire experience, subjective and objective, is founded on language. We are what (our) language makes of us.

Inherently many things
Language, along with mathematics and the natural sciences, is one of the most intensely studied ‘subjects’ for centuries, an abiding theme being whether language is inherently human. In her delightful book ‘The First Word’, Christine Kennelly describes experiments with teaching language to animals, raising some questions. According to Noam Chomsky’s concept of deep generative grammar, human beings are born with the capacity for language, a view which has been questioned recently by some researchers. This fascinating story about language will keep evolving but for the moment we can say with near 100% certainty that the capacity for abstract thinking is distinctly and uniquely human. Every noun, the concept of number and so on all testify to the human ability to abstract from the concrete to understand any concrete.

It is this ability for abstract thinking which enables humans to encounter and engage with new things and environment. We don’t have to be specifically trained for it but comes with increased mastery of language, although everyone doesn’t achieve the same level of mastery. (That is a subject for a separate discussion) This is the intelligence AI researchers want to develop in computers so that, once taught, computers will ‘use their acquired ability’ to understand newer and newer things and don’t have to be taught anew for everything. This has been the dream and goal of all great AI thinkers from Alan Turing, Marvin Minsky, Stuart Russell to now. It is natural language processing (NLP) which holds the key to unraveling the secrets of human intelligence and thus the creation of generalized AI. If have you read through all claimed breakthroughs in generalized AI, you will have noticed they are founded on language. (AI is more than NLP but my interest now is NLP.)

Language is both structured and unstructured. Linguistics teaches you to dissect language (and reasoning) through several techniques. So does literary criticism or any variation of critical studies. Unfortunately, there are many working in the field of NLP who are not even familiar with linguistics! The unstructured character of language poses challenges to interpretation – just experiment with search clues on any search engines. While some of the differences arise out of differences in the quality of the search engine, the crux of the difference is rooted in the very nature of language. Even after all these years of the wisdom of the crowds (the search engines improve with more people searching in similar ways), you do get a fair bit of indifferent results. There are times when the search engine does not offer suggestions or alternatives for a certain search clue; this is because the engine has not encountered it earlier. Quite often, the search engine fails to see the difference between a word as just a word and as a concept. This experience of search engines itself holds a clue to the almost insurmountable obstacle in language for a machine to acquire ‘generalised intelligence’ as against the ability to perform precisely defined tasks with some enhancements and modifications. This is what I keep calling a leap of faith.

This leap of faith is a gigantic challenge for one fundamental reason: language is inherently ambiguous. A single word may contain nuances that may be interpreted in many ways because language is deeply defined by culture and society and affected by the times in which we live. Words acquire different connotations in different times. Some metaphors may have become universal but many are defined by history and their times. Unless you are a part of such a milieu, understanding these metaphors is not easy; as we lead lives that move away from such milieu, these metaphors become alien to us. Most Christians will not be able to relate to Dante’s Divine Comedy because all those Biblical and other allusions are no longer an integral part of their lives. This is true of any other religious group. Our own mother tongue sounds alien to us! We see this clearly in the study of poetry. The need for translation in the same language tells us something special about how language is always evolving.

Language is at once both concrete and abstract – novels, poetry, historical narratives but also a car manual or a recipe! The same words can convey the concrete and the abstract with the ‘context’ clarifying which of the two is in operation. The context, contrary to what is taken for granted, is not objective but could be coloured by subjective considerations and hence the ‘meaning’ drawn from a set of words might well be significantly different, even in ‘one context’. There is nothing right or wrong about this; human beings are always a summation of reason, emotions, biases, prejudices – unconscious and conscious. In the literature on NLP, there is often the implicit assumption that the context will yield ‘one meaning’. The result of this unjustified assumption is either grief or disaster.

The splendor of complexity
To sump, language is arguably the most complex phenomenon known to humans. And a structured approach (usually through a Q&A technique) cannot master this complexity. Language is full of guile; she will elude you even as you thought as you have mastered it.

Computers don’t understand language; you have to convert it into digits. And that is a tough task. In 2011, Watson researchers tried to get an AI to play Jeopardy! – a game where you have to guess the question with the given answers. Some 25,000 questions were analysed, converted into 5.7 million training examples for the system – 5.7 million for 25,000! For a structured query! In all such examples, there was a failure once there were questions outside the rigidly and precisely defined problem area. Watson did not use Deep Learning as it was then relatively unknown, using instead a simple learning system. The point of making this observation has to do with computational complexity of learning systems. Deep Learning involves a higher level of computational complexity. Learning systems can be graded on their level of complexity.

All these lead me to say that a generalized AI is virtually impossible to create. Just consider the computational complexity in converting a structured query into a language that computers can comprehend and think about the level of computational complexity needed to get computers to think in a generalized, not specific manner. Should we spend so much of intellectual time and energy chasing this most elusive goal? Can’t they be put to better use trying to find answers to more urgent questions where more than adequate data is available and where patterns or the lack holds a clue to discovery – be it diagnosing cancer, drug discovery or any difficult correlation to establish? In NLP itself, there is sustained work in areas such as text compression (especially in Law), speech to text, text to speech, speech recognition which are vital and need continued support. Why is it difficult to accept that AI’s success lies in the specific? Is it worthy of pursuit only if it is an all-pervasive, generalized intelligence? The grandiose is not the only coveted intellectual goal, the simple too is.

Yes, all the great AI researchers have dreamt of this – Alan Turing, Marvin Minsky, Arthur Miller and others. All brilliant men with great accomplishments. Hence, I ask this question with a sense of trepidation and humility. Newton, with great humility, said he saw more because he was standing on the shoulders of giants. In the case of AI, perhaps researchers are seeing less precisely because they are standing on the shoulders of giants. It is a sobering thought to keep our feet firm on the ground, and step off the shoulders of giants. We will see more and farther.

Takeaways

Language is inherently many things
Many meanings may arise out of ‘one context’
Difficult to escape the elusive nature of language
Translation within the same language says profound aspects of language
Generalized AI is a ghost
Time to step off the shoulders of giants

Suggested reading
The myth of artificial intelligence – Eric J Larson
Rebooting AI – Gary Marcus and Ernest Davis
AI – Human compatible – Stuart Russell
In our own image – George Zarkadakis

PS: I will revisit the topic in the next instalment through another route: the brain