1. Introduction

The big data era has brought about the emergence of enormous amounts of data, and as a result, there is an increasing need to identify relevant information and derive insight from it. In response to this demand, knowledge graph technology has emerged and is increasingly important in realizing knowledge interconnection.

Because the building of knowledge graphs begins with a systematic description of concepts, entities, and their relationships in the objective world the correctness of information extraction of concepts, entities, and relationships is critical to the construction process.Information loss, redundancy, and overlap are often the most significant challenges to the construction of knowledge graphs. Information extraction, as the first step in knowledge graph construction, is critical to obtaining candidate knowledge units [1-2]. The completeness and accuracy of information extraction directly and explicitly affect the quality and efficiency of the subsequent knowledge graph construction steps and the quality of the final knowledge graph.

Through the method of bibliometrics, this study makes a visual analysis of the key technologies, frontier breakthroughs, and research hotspots of information extraction technology.

Through the method of literature review, this paper systematically introduces the historical background and development of information extraction. There are three sub-tasks that it can be divided into based on the technical characteristics: entity extraction, relationship extraction, and attribute extraction. Among them, each subtask is divided into specific domain-oriented and open domain-oriented according to its application field, and text-oriented and web-oriented according to its data source.

Firstly, this paper discusses the significance of information extraction research based on the framework of knowledge atlas; Then it examines the history of information extraction through the eyes of three international assessment conferences: MUC, ACE, and ICDM. Following that, the major methods of information extraction are explained in detail, including entity extraction, relationship extraction, and attribute extraction; finally, the information extraction research trend is examined.

2. Previous works on information extraction

In general, information extraction takes the text above and extracts particular information using machine learning, natural language processing (NIP), and other techniques. It then stores that information in a structured database so that users may search and use it.

Early information extraction research began in the mid-1960s, represented by two long-term projects, the linguistic string of New York University and frump of Yale University. However, it wasn't until the late 1980s that the Message Understanding Conference (MUC) was held that the study and use of information extraction progressively entered a profitable period [3]. After MUC, the National Institute of Standards and Technology (NIST) hosted the automated content extraction (ACE) assessment conference, which was a significant worldwide gathering for information extraction research from 1999 to 2008.Compared with MUC, ACE evaluation is not specific to a specific field or scenario. It adopts a set of an evaluation systems based on false positives (yes in the standard answer but not in the system output) and false positives (no in the standard answer but not in the system output). It also evaluates the cross-document processing ability of the system.

There are several types of information extraction, including entity extraction, relationship extraction, attribute extraction, and sub-tasks. There are mainly two types of entity recognition methods: specific field and open field. Domain-specific entity recognition methods mainly include some classical models, such as the maximum entropy classification model、Hidden Markov model and Conditional Random Field model, etc. In the open domain-oriented information extraction, the source of information is no longer a specific knowledge field, but a whole network of information and a large amount of Web corpus [4]. For example, KnowItAll system deals with large-scale and heterogeneous Web corpora, such as Twitter, Wikipedia, etc [5]. Due to the limitations of traditional statistical models that require a large number of corpus annotations and manual construction of a large number of features, some new methods have emerged, such as using semi-supervised algorithms, remotely supervised algorithms, self-learning method based on massive data redundancy to solve the problem of open entity extraction [6-8]. The open domain-oriented entity extraction method is often applied to the novel question-answering system based on common sense [9].

Information extraction methods are divided into natural text-oriented information extraction, Web-oriented text extraction, and social network-oriented information extraction according to the different sources of processing information. In entity recognition extraction, rule-based and statistics-based entity recognition methods are usually used to process natural language texts, which have strong pertinence and high accuracy, and can usually obtain good recognition results under manual annotation, For example, document uses a rule-based method to realize the company name as the processing object [10] The document combines the KNN classifier with the linear conditional random field (CRF) model to realize named entity recognition from short informal Twitter articles. The method based on deep learning does not need to define relevant features manually [11]. It can be applied to process natural texts in a single field. For example, literature [12] takes scientific articles as the processing object and uses a neural marking model to extract keyword phrases from scientific research articles. It can also be applied to process web data. For example, literature [13] proposes a semi-supervised system for entity recognition and distributed representation of Twitter.

In addition to natural language text and Web text, social network data is also a rich data source. Social network nodes are massive and feature a diverse set of relationships. Literature [14] proposed using a sequential joint clustering algorithm based on an unsupervised method to extract a variety of relationships in social networks containing multiple nodes.

3. Discussion

Early named entity recognition often used rule-based methods. Generally, linguistic experts first selected various features that can represent a certain type of entity according to the characteristics of the entity type to be recognized, such as the surname of a person's name, the title of a position, etc., built a limited rule template, and then completed the extraction of named entities using pattern matching [15]. Most of these systems rely on the domain knowledge of linguistic experts, which is not only time-consuming and labor-intensive but also inevitable.

With the development of machine learning, machine learning based on statistics is also continuously applied to information extraction In this method, various features of each word in the text (such as lexical features, part of speech tagging, word meaning features, etc.) are expressed as a feature vector, and then large-scale training corpus is trained through different model methods. Finally, entity recognition is carried out through the trained model. The common models are: Hmm (hidden Markov model), Me (Maxmium Entropy), SVM (support vector machine) and CRF (conditional random fields) and so on [16-19].

In recent years, with the introduction of word embedding, the application of deep learning methods in natural language processing has reached a climax. Wod2vec is the representative of word vectors. Its basic idea is to use vectors with unified dimensions to represent each word in the model [20]. This not only solves the problem of data sparsity caused by high-dimensional vector space but also integrates more semantic features into it. At the same time, heterogeneous texts can be represented by unified dimensional vector features.

Liu et al. first used CNN (Convolutional Neural Networks) to extract features automatically. It encoded sentences with word vectors and lexical features and then completed the classification with convolution, full connection, and softmax layers [21]. It improved the F1 value by 9% on the ACE 2005 dataset compared with the kernel-based method. Zeng et al. use pre-training word vectors and location features, as well as a wide maximum pooling layer behind the CNN layer [22]. Nguyen and Grishnian completely abandon the lexical features [23], allowing CNN to learn automatically, and use multi-window convolution to obtain different scales of n-gram information, achieving better results through end-to-end neural networks.

Compared with traditional machine learning methods, CNN-based methods have achieved good results, but CNN has a weak ability to extract time series features. And RNN (Recurrent Neural Network) model is suitable for extracting time series features. Zhang et al. used BRNN (Bidirectional RNN) for relationship extraction for the first time [24]. BRNN is equivalent to integrating forward and backward RNNs, which input words in sentences into two RNNs according to the forward and reverse directions respectively, and then overlay the implicit layers of the two RNNs.

Cai et al. proposed a deep learning relationship extraction module based on Shortest Dependency Path (SDP) in 2016: the bidirectional recursive convolution neural network model (BRCNN) [25]. The main idea of this paper is to model the SDP of the network syntax between two entities, encode the global information of the SDP using a dual-pass LSTM (Long Short-Term Memory), and capture the local characteristics of two words of each dependency link using CNN, to enhance the ability of entities to classify the direction of the relationship between them.

Miwa et al. first applied the neural network method to the joint model of named entity recognition and entity relationship extraction in 2016 [26]. The model is based on LSTM-RNN and executes end-to-end. The model consists of Three Representation layers. The bottom layer is the word embedding layer to complete information encoding. There are two bidirectional LSTM-RNN in the word embedding layer. One is based on word sequence for entity recognition tasks, and the other is based on dependency tree structure for relationship extraction. The two parts share encoding information and stack to form an overall model. As part of the input of the latter structure, the former output and the hidden layer make entity recognition and extraction interact.

Katiyar et al. combined Attention, an attention mechanism, with BiLSTM for named entity recognition and relationship extraction in 2017 [27]. The model draws on the model of Miwa et al. and improves the disadvantage of the original model depending on the intersectional sequence, dependency tree, etc [25]. The model has an input layer with one embedded word representation, two output layers, an entity for output identification, and a relationship classification using the attention model.

In 2018, Devlin et al. proposed the BERT (Bidirectional Encoder Representations from Transformers) model. BERT belongs to the pre-training language model [28]. The so-called pre-training model is to pre-train the model with a large number of customized text so that the model can acquire general language knowledge, and then perform Fine-tuning phase training according to downstream tasks so that the model parameters can be fine-tuned according to specific task requirements and domain knowledge.

In recent years, the emergence of pre-training models such as GPT and BERT has made the Q&A reading comprehension task a good downstream task of theirs [28-29]. Simply reconstructing the original network structure and fine-tuning can get good results. Wang et al. improved their performance on the SQuAD dataset by using multi-paragraph prediction based on the original BERT [30-31]. Alberti and others improved on BERT and SQuAD and applied them to a more difficult question and answer data set, NQ [32-33].

At present, information extraction methods based on deep-learning have made good progress, but there are still many aspects worth further study. First, the deep-learning model is good at processing single-sentence semantic information, but in practice, many entity relationships are expressed by multiple statements together, which requires the model to comprehensively understand, memorize and infer multiple statements in the document, and extract document-level relationships. Secondly, the current research on information extraction is mostly focused on the preset set of extraction tasks, but future applications will be open-domain-oriented information extraction. Therefore, it is necessary to continuously explore how to automatically discover new entity relationships and their facts in the open domain. Finally, current research is often limited to single-language text information, and human beings can process multiple information synthetically when receiving information. Therefore, it is necessary to explore how to extract relationships using multilingual text, sound and video information synthetically.

4. Conclusion

This paper first introduces the concept of information extraction based on the concept of knowledge map and the construction of technical framework. Then it briefly introduces the history of information extraction through three international evaluation conferences and three development stages (rule-based stage, statistical learning stage, deep-learning stage). Subsequently, the latest development and a series of cases of key information extraction technologies combined with CNN, RNN, LSTM, BERT and other deep-learning algorithms are introduced in detail. Finally, some problems and research trends that need to be solved for future information extraction are analyzed.