What You Need to Know about Natural Language Processing
You have heard the term thrown around the office and at tech events; you know it’s the latest and greatest of data science processes, but what is Natural Language Processing really? One of Welcome’s Data Intelligence Analysts Ben Nichols provided a breakdown:
What is it?
Bodies of text, conversations, pictures - all of these are “data” that machines can potentially analyze. However, computers and machines can only understand data in specific forms, and human data is not one of those forms. That requires us, as humans, to manually break down this human data into a language machines understand. Natural Language Processing is a discipline that focuses on enabling machines to analyze written human text on its own without the help of humans.
Basically, NLP is an area of study to figure out how machines can better understand humans.
How is it used?
Every single day, people use an app or feature that is a result of NLP. Thanks to NLP, Siri can understand your verbal request to “call Mom” or “book a reservation”. Google Translate uses NLP to intuitively understand the grammatical structure of sentences so it can translate it from English to Español.
At Welcome, we have thousands of chats that need to be read through and analyzed. Traditionally we offered a personalized service where humans comb through every conversation to interpret text and manually annotate. However, by leveraging NLP we can scale this process, allowing for a deeper analysis of the underlying drivers of those conversations.
When did it come into being?
The first experiment in NLP was in 1954, a lot farther into the past than one would expect: a Georgetown experiment fully translated sixty Russian sentences into English. The author of this study believed that machine translation would be complete in three to five years. In reality, it took much longer than that.
"The majority of the algorithms used today were developed in the 1970s, but we didn't have the technology available to do anything with it, so it was more theoretical,” said Ben. “NLP is capitalizing on the technology developed, so we can use those algorithms more effectively.”
Why is it important?
Improve quantity without sacrificing quality. “One of the biggest opportunities NLP presents is the ability to scale traditionally human processes,” said Ben. “By broadening the input we can provide machines, we can make computers inherently understand what humans write, draw or say.”
Stanford Professor Dan Jurafsky provides a tutorial: