Natural Language Processing (NLP) – What is It?

We got used to communicating with a computer on its terms. We use a mouse, keyboard, and specific commands that sometimes sound unnatural for us but they are easy to understand for our computers or mobile phones. This situation is changing rapidly. Services like SIRI, Cortana, and Google Now allow us to communicate with the device via speech. Natural language processing makes it possible to believe that we will soon be able to communicate with computers using only the voice interface. We will not have to adapt to the “language” of the machine because it will be able to understand ours.

What is Natural Language Processing?

Natural language processing is a field of research that examines how computers can be taught to understand and use natural (i.e. human) language. It began in 1954, as part of the Georgetown–IBM experiment, when researchers managed to use computers to translate few dozens of sentences from Russian into English. The research was limited and was based only on basic grammatical rules and a small dictionary. Nevertheless, it raised a lot of interest. Despite initial enthusiasm, research of the NLP slowed down until the 1980s. It wasn’t until then that the increase in computational power and the use of machine learning allowed for the significant development of NLP. Today it is one of the most rapidly developing areas of AI and is finding more and more practical applications in everyday life.

Natural language processing can be broadly divided into two areas. One is the natural language understanding – the ability to take some spoken or typed sentences and getting to know what they mean. The other is natural language generation – finding a way to express something in natural, human language.

Stages of Natural Language Understanding

The process of understanding the language consists of four successive stages. We begin with the speech recognition. Input data is the human voice recorded by the microphone. It is then transformed from an analog signal to digital data and compared with the templates in the system database to find the matching words.

The next step of NL understanding is the syntactic analysis, that is processing the string of words regarding grammar rules. The system verifies that the sentence is grammatically correct and then processes it into a parse tree representing the syntactic structure of the phrase. The third stage is semantic analysis. Individual words are analyzed in terms of meaning. Then the system generates the general sense of the sentence based on the syntax.

The last step is pragmatics. It’s the phase where most problems arise in understanding the context of utterance, dealing with pronouns or ambiguity of words and sentences. Full understanding of statement sometimes requires knowledge of the relationship between speakers or their intentions.

Natural Language Generation

Natural language generation is a way for a computer to talk back. It is the reversed process of understanding. The first step is content determination – deciding what to say. The second one is building the utterance – deciding how to put concepts into the words considering their meaning and grammar rules.

The last stage is realization – creating a particular text according to the rules of syntax, morphology, orthography, and finally speech synthesis. It may be solely based on the templates using the recording of human speech or more complex, by combining phonemes in the phonetic spelling of every word.

Applications of Natural Language Processing

NLP is an extremely promising AI area. Voice interface is certainly one of the most interesting uses of this technology, after all, who would not like to talk to his computer like to C3PO from Star Wars. But there are plenty more practical applications of NLP. The first thing that comes to mind is the automatic translation of texts which already stands at a high level. NLP can also be useful for information retrieval – finding among documents the ones that are relevant to the query or extracting structured data from unstructured text, as well as more simple tasks like spelling and grammar checkers.