基于神經(jīng)網(wǎng)絡的中文詞法分析系統(tǒng)的研究與實現(xiàn)
發(fā)布時間：2018-08-14 08:48
【摘要】：本論文的研究目的是構建一套基于神經(jīng)網(wǎng)絡的中文詞法分析(中文分詞,詞性標注和命名實體識別)系統(tǒng),主要研究內容包含兩點,其一是研究適合中文詞法分析任務的具體模型,其二是研究如何將其良好地實現(xiàn)。首先我們多維度地簡要介紹了中文詞法分析的各個任務,隨后調研了當下已有的詞法分析系統(tǒng)。接著我們將用于序列標注的神經(jīng)網(wǎng)絡結構拆分為輸入層、表示學習層和標簽預測層,并逐層展開介紹。隨后本文以實驗為出發(fā)點,探究了不同的輸入特征和模型結構在各任務上的效果,確定了適合各個任務的神經(jīng)網(wǎng)絡模型。我們得到的各模型結構不盡相同,但都以雙向LSTM結構作為表示學習方法,同時融合手工特征或未標注數(shù)據(jù)的信息。最后,我們介紹了系統(tǒng)實現(xiàn)的代碼結構,并完成系統(tǒng)的速度評估。本論文的研究成果主要包含兩點。第一點是通過實驗確定了適合各中文詞法分析任務的具體神經(jīng)網(wǎng)絡結構。我們選擇LTP作為基準線模型,以LTP使用的數(shù)據(jù)集作為實驗數(shù)據(jù)集。在中文分詞任務上,我們的模型在開發(fā)集和測試集上的F1值比LTP分別高0.33、0.48個百分點;在詞性標注任務上,最優(yōu)模型在開發(fā)集上的Accuracy比基準線高0.2個百分點,測試集上高0.22個點;在命名實體識別上,我們確定的模型在開發(fā)集和測試集上的F1值比LTP提升了2.57和0.57個百分點。第二個研究成果體現(xiàn)在系統(tǒng)實現(xiàn)上。我們用清晰地代碼結構實現(xiàn)了上述神經(jīng)網(wǎng)絡模型,獲得了一套可用的中文詞法分析系統(tǒng)。
[Abstract]:The purpose of this thesis is to construct a Chinese lexical analysis system (Chinese word segmentation, part of speech tagging and named entity recognition) based on neural network. One is to study the specific model suitable for Chinese lexical analysis, and the other is to study how to implement it well. Firstly, we briefly introduce the tasks of Chinese lexical analysis, and then investigate the existing lexical analysis systems. Then we divide the neural network structure used for sequence tagging into input layer, represent learning layer and label prediction layer, and introduce them layer by layer. Then based on the experiment, this paper explores the effects of different input characteristics and model structures on each task, and determines the neural network model suitable for each task. The structure of each model is different, but the bidirectional LSTM structure is used as the representation learning method, and the information of manual feature or unlabeled data is fused at the same time. Finally, we introduce the code structure of the system, and complete the speed evaluation of the system. The research results of this paper mainly contain two points. The first point is to determine the specific neural network structure suitable for each Chinese lexical analysis task through experiments. We choose LTP as the baseline model and the data set used by LTP as the experimental data set. On the task of Chinese word segmentation, the F1 value of our model is 0.33% higher than that of LTP on the development set and the test set, and the Accuracy of the optimal model is 0.2% higher than the baseline on the part of speech tagging task, and the test set is 0.22 points higher than the test set. In the named entity recognition, the F1 value of the model is 2.57% and 0.57% higher than that of LTP in the development set and test set. The second research result is embodied in the system implementation. We implement the neural network model with clear code structure and obtain a set of available Chinese lexical analysis system.
【學位授予單位】：哈爾濱工業(yè)大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP391.1;TP183
本文編號：2182300
[Abstract]:The purpose of this thesis is to construct a Chinese lexical analysis system (Chinese word segmentation, part of speech tagging and named entity recognition) based on neural network. One is to study the specific model suitable for Chinese lexical analysis, and the other is to study how to implement it well. Firstly, we briefly introduce the tasks of Chinese lexical analysis, and then investigate the existing lexical analysis systems. Then we divide the neural network structure used for sequence tagging into input layer, represent learning layer and label prediction layer, and introduce them layer by layer. Then based on the experiment, this paper explores the effects of different input characteristics and model structures on each task, and determines the neural network model suitable for each task. The structure of each model is different, but the bidirectional LSTM structure is used as the representation learning method, and the information of manual feature or unlabeled data is fused at the same time. Finally, we introduce the code structure of the system, and complete the speed evaluation of the system. The research results of this paper mainly contain two points. The first point is to determine the specific neural network structure suitable for each Chinese lexical analysis task through experiments. We choose LTP as the baseline model and the data set used by LTP as the experimental data set. On the task of Chinese word segmentation, the F1 value of our model is 0.33% higher than that of LTP on the development set and the test set, and the Accuracy of the optimal model is 0.2% higher than the baseline on the part of speech tagging task, and the test set is 0.22 points higher than the test set. In the named entity recognition, the F1 value of the model is 2.57% and 0.57% higher than that of LTP in the development set and test set. The second research result is embodied in the system implementation. We implement the neural network model with clear code structure and obtain a set of available Chinese lexical analysis system.
【學位授予單位】：哈爾濱工業(yè)大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP391.1;TP183
【參考文獻】
相關期刊論文 前7條
1 黃昌寧;靳光瑾;;從賓州中文樹庫觀察三個漢語語法問題[J];語言科學;2013年02期
2 張梅山;鄧知龍;車萬翔;劉挺;;統(tǒng)計與詞典相結合的領域自適應中文分詞[J];中文信息學報;2012年02期
3 王麗杰;車萬翔;劉挺;;基于SVMTool的中文詞性標注[J];中文信息學報;2009年04期
4 黃昌寧;趙海;;中文分詞十年回顧[J];中文信息學報;2007年03期
5 蘇祺,昝紅英,胡景賀,項錕;詞性標注對信息檢索系統(tǒng)性能的影響[J];中文信息學報;2005年02期
6 周強;漢語句法樹庫標注體系[J];中文信息學報;2004年04期
7 周強;規(guī)則和統(tǒng)計相結合的漢語詞類標注方法[J];中文信息學報;1995年03期
，本文編號：2182300
本文鏈接：http://sikaile.net/kejilunwen/ruanjiangongchenglunwen/2182300.html
最近更新
教材專著
