基于上下文的維度情感識(shí)別方法研究
發(fā)布時(shí)間：2018-07-29 07:13
【摘要】：情感在人們?nèi)粘＝涣髦邪缪葜匾慕巧?豐富的情感有助于說(shuō)話人表達(dá)自己的思想。維度情感可以描述復(fù)雜微妙且連續(xù)的情感狀態(tài),它將不同的情感狀態(tài)表征為一個(gè)連續(xù)的情感空間中不同的點(diǎn)。人類的情感表達(dá)是連續(xù)的、多模態(tài)的,因此,在維度情感識(shí)別中,基于上下文的情感識(shí)別方法越來(lái)越受到研究者的關(guān)注�，F(xiàn)有的基于上下文的情感識(shí)別方法主要集中在情感特征上學(xué)習(xí)上下文信息,忽略了情感狀態(tài)上下文信息的學(xué)習(xí),且很少考慮模態(tài)之間的情感上下文。因此,本文主要通過(guò)情感時(shí)間上下文和情感模態(tài)上下文兩個(gè)方面來(lái)研究上下文信息對(duì)維度情感識(shí)別的作用。情感時(shí)間上下文是指情感在表達(dá)過(guò)程中隨時(shí)間變化的規(guī)律,包括情感特征和情感狀態(tài)的連續(xù)變化,情感模態(tài)上下文是指多個(gè)模態(tài)之間所表現(xiàn)的情感信息的相互關(guān)聯(lián)性。充分利用這兩種上下文信息有助于提高維度情感識(shí)別準(zhǔn)確率。具體研究?jī)?nèi)容如下:1)提出基于雙向長(zhǎng)短時(shí)記憶網(wǎng)絡(luò)的層次情感時(shí)間上下文學(xué)習(xí)方法:該方法包含三個(gè)步驟。首先,對(duì)輸入的低層特征通過(guò)前饋神經(jīng)網(wǎng)絡(luò)學(xué)習(xí)得到高層特征,這樣可以消除低層特征的不穩(wěn)定性,從而得到表征能力更好的高層特征。然后,在高層特征上通過(guò)雙向長(zhǎng)短時(shí)記憶網(wǎng)絡(luò)學(xué)習(xí)情感特征序列的情感時(shí)間上下文信息,利用此信息對(duì)情感狀態(tài)進(jìn)行初步的識(shí)別。最后,通過(guò)無(wú)監(jiān)督學(xué)習(xí)方法得到情感標(biāo)簽序列的情感時(shí)間上下文信息,利用此信息對(duì)上階段得到初步識(shí)別結(jié)果做最終識(shí)別。本方法通過(guò)學(xué)習(xí)情感特征序列和情感標(biāo)簽序列的情感時(shí)間上下文信息,從而充分利用情感狀態(tài)表達(dá)的連續(xù)性特點(diǎn)進(jìn)行維度情感識(shí)別。在AVEC2015數(shù)據(jù)集上的實(shí)驗(yàn)結(jié)果表明,利用情感特征和情感標(biāo)簽兩種情感時(shí)間上下文得到的識(shí)別結(jié)果要好于僅利用特征的情感時(shí)間上下文得到的識(shí)別結(jié)果。2)提出基于注意力模型的動(dòng)態(tài)情感模態(tài)上下文學(xué)習(xí)方法:該方法包含兩個(gè)步驟,首先采用上一方法分別基于視頻與音頻數(shù)據(jù)的情感時(shí)間上下文信息對(duì)維度情感狀態(tài)進(jìn)行初步識(shí)別,分別得到基于單模態(tài)的維度情感識(shí)別結(jié)果。然后,基于注意力模型進(jìn)行情感模態(tài)上下文學(xué)習(xí)。情感模態(tài)上下文學(xué)習(xí)過(guò)程中,在每一時(shí)刻對(duì)每個(gè)模態(tài)數(shù)據(jù)通過(guò)注意力模型實(shí)時(shí)地計(jì)算出各自模態(tài)的注意力信號(hào)量,將該注意力信號(hào)量作為相應(yīng)模態(tài)對(duì)情感識(shí)別的權(quán)重,進(jìn)而動(dòng)態(tài)地計(jì)算出當(dāng)前時(shí)刻的模態(tài)上下文向量。最后將學(xué)習(xí)得到模態(tài)上下文向量輸入到雙向長(zhǎng)短時(shí)記憶網(wǎng)絡(luò)進(jìn)行維度情感識(shí)別。本方法能夠動(dòng)態(tài)地學(xué)習(xí)情感模態(tài)上下文信息。在AVEC2015和RECOLA兩個(gè)數(shù)據(jù)集上的實(shí)驗(yàn)結(jié)果表明,與基于單模態(tài)的識(shí)別方法相比,該方法能夠提高識(shí)別準(zhǔn)確性,而且通過(guò)注意力模型動(dòng)態(tài)地學(xué)習(xí)情感模態(tài)上下文得到的識(shí)別結(jié)果好于傳統(tǒng)的基于線性方法學(xué)習(xí)情感模態(tài)上下文得到的識(shí)別結(jié)果。3)設(shè)計(jì)并實(shí)現(xiàn)基于上下文的維度情感識(shí)別原型系統(tǒng):采用PyQt實(shí)現(xiàn)了系統(tǒng)的圖形操作界面,基于Python、Numpy、CUDA和Theano實(shí)現(xiàn)了系統(tǒng)的算法。原型系統(tǒng)包括數(shù)據(jù)處理、情感時(shí)間上下文學(xué)習(xí)、情感模態(tài)上下文學(xué)習(xí)三個(gè)模塊。通過(guò)該原型系統(tǒng)的實(shí)現(xiàn)來(lái)驗(yàn)證本文所提方法的可用性。
[Abstract]:Emotion plays an important role in people's daily communication. The rich emotion helps the speaker express his thoughts. The dimension emotion can describe the complex and continuous emotional state. It characterizing different emotional states as different points in a continuous emotional space. Human emotion expression is continuous and multimodal. Therefore, in dimension emotion recognition, the context based emotion recognition method has attracted more and more attention. The existing context based emotion recognition methods mainly focus on the learning context information on the emotional features, ignore the learning of the emotional state context information, and seldom consider the emotional context between the modes. This paper studies the effect of context information on dimensional emotion recognition mainly through two aspects of emotional time context and emotional modal context. Emotional time context refers to the regularity of emotion in the process of expression, including the continuous transformation of emotional and emotional states, and emotional modal context refers to multiple modes. The interrelation between the emotional information expressed between them. The full use of these two contextual information helps to improve the accuracy of dimension emotion recognition. The specific research contents are as follows: 1) a hierarchical affective time context learning method based on the bidirectional long short memory network is proposed: this method contains three steps. First, the low layer characteristics of the input are passed. The feedforward neural network learns the high-level features so that the instability of the low layer features can be eliminated and the high-level features with better characterization are obtained. Then, the emotional time context information is learned from the emotional feature sequence by the bidirectional long short memory network on the high level feature, and the emotional state is identified by this information. Finally, the emotional time context information of the emotional label sequence is obtained by the unsupervised learning method. This information is used to identify the initial recognition results at the upper stage. This method can make full use of the continuity of emotional state expression by learning the emotional time context information of the emotional feature sequence and the emotional label sequence. The experimental results on the AVEC2015 data set show that the recognition results obtained from two emotional time contexts using emotional features and emotional labels are better than the recognition results that only use the emotional time context of the feature.2) to propose a dynamic affective modal context learning method based on attention mode: this method: The method consists of two steps. First, the first method is based on the emotional time context information of the video and audio data to identify the emotional state of the dimension, and the results of emotional recognition based on single mode are obtained respectively. Then, the emotional modal context learning is carried out based on the attention model. The emotional modal context has been studied. In the process, each modal data is calculated in real time by the attention model of each modal data, and the amount of attention signal is used as the weight of the corresponding modal for emotion recognition, and then the modal context vector of the current moment is calculated dynamically. This method can dynamically learn emotional modal context information. The experimental results on two data sets in AVEC2015 and RECOLA show that the method can improve the accuracy of recognition compared with the single mode based recognition method and learn the emotional mode dynamically through the attention model. The recognition result of context is better than the traditional recognition result.3 based on linear method learning emotional mode context.) design and implement a context based dimension emotion recognition prototype system: using PyQt to implement the system graphical interface, based on Python, Numpy, CUDA and Theano implementation of the system algorithm. Prototype system package It includes three modules, including data processing, emotional time context learning and emotional modal context learning. Through the implementation of the prototype system, the availability of the proposed method is verified.
【學(xué)位授予單位】：江蘇大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP18
本文編號(hào)：2151916
[Abstract]:Emotion plays an important role in people's daily communication. The rich emotion helps the speaker express his thoughts. The dimension emotion can describe the complex and continuous emotional state. It characterizing different emotional states as different points in a continuous emotional space. Human emotion expression is continuous and multimodal. Therefore, in dimension emotion recognition, the context based emotion recognition method has attracted more and more attention. The existing context based emotion recognition methods mainly focus on the learning context information on the emotional features, ignore the learning of the emotional state context information, and seldom consider the emotional context between the modes. This paper studies the effect of context information on dimensional emotion recognition mainly through two aspects of emotional time context and emotional modal context. Emotional time context refers to the regularity of emotion in the process of expression, including the continuous transformation of emotional and emotional states, and emotional modal context refers to multiple modes. The interrelation between the emotional information expressed between them. The full use of these two contextual information helps to improve the accuracy of dimension emotion recognition. The specific research contents are as follows: 1) a hierarchical affective time context learning method based on the bidirectional long short memory network is proposed: this method contains three steps. First, the low layer characteristics of the input are passed. The feedforward neural network learns the high-level features so that the instability of the low layer features can be eliminated and the high-level features with better characterization are obtained. Then, the emotional time context information is learned from the emotional feature sequence by the bidirectional long short memory network on the high level feature, and the emotional state is identified by this information. Finally, the emotional time context information of the emotional label sequence is obtained by the unsupervised learning method. This information is used to identify the initial recognition results at the upper stage. This method can make full use of the continuity of emotional state expression by learning the emotional time context information of the emotional feature sequence and the emotional label sequence. The experimental results on the AVEC2015 data set show that the recognition results obtained from two emotional time contexts using emotional features and emotional labels are better than the recognition results that only use the emotional time context of the feature.2) to propose a dynamic affective modal context learning method based on attention mode: this method: The method consists of two steps. First, the first method is based on the emotional time context information of the video and audio data to identify the emotional state of the dimension, and the results of emotional recognition based on single mode are obtained respectively. Then, the emotional modal context learning is carried out based on the attention model. The emotional modal context has been studied. In the process, each modal data is calculated in real time by the attention model of each modal data, and the amount of attention signal is used as the weight of the corresponding modal for emotion recognition, and then the modal context vector of the current moment is calculated dynamically. This method can dynamically learn emotional modal context information. The experimental results on two data sets in AVEC2015 and RECOLA show that the method can improve the accuracy of recognition compared with the single mode based recognition method and learn the emotional mode dynamically through the attention model. The recognition result of context is better than the traditional recognition result.3 based on linear method learning emotional mode context.) design and implement a context based dimension emotion recognition prototype system: using PyQt to implement the system graphical interface, based on Python, Numpy, CUDA and Theano implementation of the system algorithm. Prototype system package It includes three modules, including data processing, emotional time context learning and emotional modal context learning. Through the implementation of the prototype system, the availability of the proposed method is verified.
【學(xué)位授予單位】：江蘇大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP18
【參考文獻(xiàn)】
相關(guān)期刊論文 前2條
1 毛啟容;白李娟;王麗;詹永照;;基于情感上下文的語(yǔ)音情感推理算法[J];模式識(shí)別與人工智能;2014年09期
2 國(guó)玉晶;劉剛;劉健;郭軍;;基于環(huán)境特征的語(yǔ)音識(shí)別置信度研究[J];清華大學(xué)學(xué)報(bào)(自然科學(xué)版);2009年S1期
，本文編號(hào)：2151916
本文鏈接：http://sikaile.net/shoufeilunwen/xixikjs/2151916.html
最近更新
教材專著
