【摘要】：近年來,深度學(xué)習(xí)越來越廣泛地應(yīng)用于自然語言處理領(lǐng)域,人們提出了諸如循環(huán)神經(jīng)網(wǎng)絡(luò)(RNN)等模型來構(gòu)建文本表達(dá)并解決文本分類等任務(wù)。長(zhǎng)短時(shí)記憶(long short term memory,LSTM)是一種具有特別神經(jīng)元結(jié)構(gòu)的RNN。LSTM的輸入是句子的單詞序列,模型對(duì)單詞序列進(jìn)行掃描并最終得到整個(gè)句子的表達(dá)。然而,常用的做法是只把LSTM在掃描完整個(gè)句子時(shí)得到的表達(dá)輸入到分類器中,而忽略了掃描過程中生成的中間表達(dá)。這種做法不能高效地提取一些局部的文本特征,而這些特征往往對(duì)決定文檔的類別非常重要。為了解決這個(gè)問題,該文提出局部化雙向LSTM模型,包括MaxBiLSTM和ConvBiLSTM。MaxBiLSTM直接對(duì)雙向LSTM的中間表達(dá)進(jìn)行max pooling。ConvBiLSTM對(duì)雙向LSTM的中間表達(dá)先卷積再進(jìn)行max pooling。在兩個(gè)公開的文本分類數(shù)據(jù)集上進(jìn)行了實(shí)驗(yàn)。結(jié)果表明,局部化雙向LSTM尤其是ConvBiLSTM相對(duì)于LSTM有明顯的效果提升,并取得了目前的最優(yōu)結(jié)果。
[Abstract]:In recent years, in-depth learning is more and more widely used in the field of natural language processing. People put forward some models such as cyclic neural network (RNN) to construct text representation and solve the task of text classification. Long and short memory (long short term memory,LSTM) is a kind of RNN.LSTM with special neuronal structure. The input of the RNN.LSTM is the word sequence of the sentence. The model scans the sequence of words and finally obtains the expression of the whole sentence. However, the usual approach is to input the expression obtained by LSTM into the classifier after scanning the whole sentence, while ignoring the intermediate expression generated in the scanning process. This approach can not efficiently extract some local text features, which are often very important in determining the classification of documents. In order to solve this problem, this paper proposes a localized bidirectional LSTM model, which includes MaxBiLSTM and ConvBiLSTM.MaxBiLSTM directly implementing max pooling.ConvBiLSTM for the intermediate expression of bidirectional LSTM and convolution then max pooling. for the intermediate expression of bidirectional LSTM. Experiments are carried out on two published text categorization data sets. The results show that the localization of bidirectional LSTM, especially ConvBiLSTM, has a significant improvement over LSTM, and the best results are obtained.
【作者單位】： 中國科學(xué)院計(jì)算技術(shù)研究所;中國科學(xué)院大學(xué);
【基金】：973基金項(xiàng)目(2014CB340401,2012CB316303) 國家自然科學(xué)基金(6122010,61472401,61433014,61425016,61203298) 中國科學(xué)院青年創(chuàng)新促進(jìn)會(huì)(2014310,2016102)
【分類號(hào)】：TP18;TP391.1
，

本文編號(hào)：2308635