As the research direction of language information processing technology, natural language processing has always been one of the core topics in the field of artificial intelligence. In daily life, we sometimes encounter some ambiguous or puzzling sentence examples, which make people intuitively understand that it is too difficult for computers to understand human language. This paper introduces "what is the difficulty of NLP" from four aspects: the essence and key of natural language understanding, and the characteristics and difficulties of natural language understanding.

The total number of words in this article is 62 14, and the reading time is about 2 1 minute.

Brief introduction of the author

Liu Zhiyuan is an associate professor in the Computer Science Department of Tsinghua University. His research interests include natural language processing, knowledge map and semantic computing, social computing and computational social science.

First, the nature of natural language understanding is structural prediction.

To understand the difficulty of natural language understanding, we must first look at the nature of natural language understanding tasks. As one of the three kinds of information (voice, vision and language) concerned by artificial intelligence, natural language text is a typical unstructured data, which is composed of a series of language symbols (such as Chinese characters). In order to understand the ideographic meaning of natural language, it is necessary to establish a prediction of the semantic structure behind unstructured texts. Therefore, many tasks of natural language understanding include but are not limited to Chinese word segmentation, part-of-speech tagging, named entity recognition, * * * * anaphora resolution, syntactic analysis, semantic role tagging and so on. Are all about predicting the specific semantic structure behind the text sequence. For example, Chinese word segmentation is to add spaces or other symbols to sentences without spaces to mark the boundaries of each word in the sentence, which is equivalent to adding some structural and semantic information to this text sequence.