Tag Archives: computer science

Phonemes From Our Phones: Natural Language Processing

Imagine creating a step-by-step list to understanding language. That’s the daunting task that computer scientists face when developing algorithms for Natural Language Processing (or NLP).

First of all, there are roughly 150,000 words in the English language (Oxford English Dictionaries, n.d.), and many of them are synonyms or words with multiple meanings. Think of words like “back” or “miss,” which would be maddening to understand for an English learner: “I went back to the back of the room and laid on my back.” or “Miss, even though you missed the point, I will miss you when you’re gone.”

After parsing through those tens of thousands of words and all their associated meanings and variations, there arises the issue of dialects. English in Australia sounds different than English in Ireland, which sounds different than English in Canada. Moreover, even within a country, there can be multiple dialects: in the United States, consider how different people sound from Mississippi compared to Michigan, or Massachusetts compared to New Mexico. This blog post by internet linguist Gretchen McCulloch dives into some of these issues, and raises another interesting point: how do we teach computers to read, pronounce, and/or understand abbreviations and the new forms of English specific to internet communication, such as “lol,” “omg,” and “smh”?

Other issues such as tone and inflection can drastically change the meaning of a sentence when spoken aloud. I found one example from the Natural Language Processing video from Crash Course Computer Science to be especially powerful, where they took a simple sentence “She saw me” and changed the meaning 3 times by altering the inflection (Brungard, 2017):

“Who saw you?” … “She saw me.”

“Who did she see? … “She saw me.”

“Did she hear you or see you?” … “She saw me.”

I want to take a brief moment to appreciate the Crash Course Computer Science video series. That series takes extremely dense and complex topics and packages them into brief, comprehensive, lighthearted videos with delightfully animated (and often topical) visual aids and graphics. I will undoubtedly be returning to them for many more computer science-related quandaries. 

Anyway, all these different obstacles that make natural language processing difficult to program and code for computer scientists (vocabulary, synonyms, dialects, inflection, tone, etc.) change from language to language. So designing for Spanish or Chinese or Arabic will have many similar obstacles as English, while also presenting new and different hurdles unique to each language and its particular nuances. Luckily for us, companies like Google are rolling out huge supercomputers like BERT, “with 24 Transformer blocks, 1024 hidden layers, and 340M parameters,” that are capable of processing (and, in effect, “learning”) billions of words across multiple languages and “even surpassing human performance in the challenging area of question answering” (Peng, 2018). This helps explain why “talking robots” like Siri and Alexa have become less creepy-sounding, more efficient, and much more popular in recent years. 

Obviously, NLP is a huge undertaking for computer scientists, and there is still plenty of work to be done before computers can consistently, efficiently, and seamlessly understand and interact with human language. But with the sheer amount of language and linguistic data available online now (and increasing at an exponential rate), we may look back on this conversation in 5-10 years and laugh. And the computers might laugh with us.

 

References

Brungard, B. (2017). Natural Language Processing [Video] (Vol. 36). PBS Digital Studios. Retrieved from https://www.youtube.com/watch?v=fOvTtapxa9c

How many words are there in the English language? (n.d.). Oxford English Dictionaries. Retrieved from https://en.oxforddictionaries.com/explore/how-many-words-are-there-in-the-english-language/

McCulloch, G. (2017). Teaching computers to recognize ALL the Englishes. Retrieved from https://allthingslinguistic.com/post/150556285220/teaching-computers-to-recognize-all-the-englishes

Peng, T. (2018, October 16). Best NLP Model Ever? Google BERT Sets New Standards in 11 Language Tasks. Medium. Retrieved from https://medium.com/syncedreview/best-nlp-model-ever-google-bert-sets-new-standards-in-11-language-tasks-4a2a189bc155