基于卷積—長(zhǎng)短時(shí)記憶神經(jīng)網(wǎng)絡(luò)的時(shí)序信號(hào)多粒度分析處理方法研究
發(fā)布時(shí)間：2018-07-28 12:19
【摘要】：時(shí)序信號(hào)是一種極其重要的信號(hào),是指如語(yǔ)音信號(hào)、生物電信號(hào)、雷達(dá)和聲納信號(hào)、機(jī)械振動(dòng)和地震信號(hào)[1]等等這樣的頻率、幅值隨著時(shí)間的變化而不斷改變的多成分信號(hào)。時(shí)序信號(hào)具有非線性和非平穩(wěn)的特點(diǎn),目前的絕大多數(shù)研究中都是基于信號(hào)是短時(shí)平穩(wěn)的假設(shè),特征的提取主要以頻域特征為主,分析的層面和粒度相對(duì)單一。而且信號(hào)中極為重要的大部分時(shí)序信息被忽略,極大地影響了對(duì)時(shí)變信號(hào)信息的提取的能力,限制了其在實(shí)際應(yīng)用中性能的提升。本文針對(duì)時(shí)序信號(hào)中時(shí)序信息的提取和建模問(wèn)題,借鑒人腦認(rèn)知過(guò)程中能夠自動(dòng)優(yōu)選和整合多粒度、多時(shí)段和多層次特征的能力,提出了多粒度特征的提取和融合方法框架,我們將信號(hào)按照幀、段和全局三個(gè)粒度進(jìn)行特征的提取,這樣既保留了現(xiàn)有方法普遍采用的全局特征,又增加了幀粒度和段粒度這兩個(gè)包含信號(hào)中時(shí)序信息的動(dòng)態(tài)特征,有效地從多個(gè)角度對(duì)時(shí)序信號(hào)中的信息進(jìn)行了提取,對(duì)信號(hào)中信息的表達(dá)能力也更加豐富。在段粒度的劃分上,我們參考人腦在認(rèn)知活動(dòng)中的規(guī)律來(lái)進(jìn)行窗長(zhǎng)的劃分。之后,我們將三個(gè)粒度的特征統(tǒng)一在幀的層面上按照時(shí)間的順序進(jìn)行了融合,再利用對(duì)時(shí)序信息建模能力比較強(qiáng)的LSTM神經(jīng)網(wǎng)絡(luò)模型來(lái)進(jìn)行分類。在多粒度特征的具體實(shí)現(xiàn)上,我們采用了兩個(gè)方法。一是利用傳統(tǒng)的時(shí)頻分析方法對(duì)時(shí)序信號(hào)進(jìn)行幀特征的提取,再利用高斯函數(shù)組在段粒度窗下對(duì)幀特征進(jìn)行卷積計(jì)算得到段特征,全局特征則是通過(guò)對(duì)所有的幀特征進(jìn)行統(tǒng)計(jì)計(jì)算得到。另一個(gè)方法是結(jié)合目前在各個(gè)領(lǐng)域都有突破性進(jìn)展的深度學(xué)習(xí)技術(shù),借助卷積神經(jīng)網(wǎng)絡(luò)可以在原始數(shù)據(jù)上進(jìn)行端到端的信息提取的能力,以及在多個(gè)層級(jí)完成特征提取的特點(diǎn),來(lái)對(duì)時(shí)序信號(hào)完成多粒度的特征提取,提出了C-LSTM的網(wǎng)絡(luò)結(jié)構(gòu)。我們將待分析的時(shí)序原始信號(hào)直接輸入到深度卷積網(wǎng)絡(luò)中,通過(guò)預(yù)先設(shè)置好的卷積核在信號(hào)上進(jìn)行滑動(dòng)卷積,在淺層CNN中獲得幀粒度特征,同時(shí)繼續(xù)對(duì)幀粒度特征用更高層的CNN進(jìn)一步加工,分別在中層和高層CNN輸出段粒度以及全局粒度的特征。最后將三個(gè)粒度的特征信息在幀層面上按照時(shí)序進(jìn)行整合,得到多粒度融合特征,并利用長(zhǎng)短時(shí)記憶網(wǎng)絡(luò)對(duì)時(shí)序信息進(jìn)行建模與分類。最后,我們將所提出的方法框架和網(wǎng)絡(luò)結(jié)構(gòu)模型分別在語(yǔ)音信號(hào)上的語(yǔ)音情感識(shí)別分類問(wèn)題以及腦電信號(hào)上的運(yùn)動(dòng)想象信號(hào)分類識(shí)別問(wèn)題進(jìn)行了實(shí)驗(yàn)。在語(yǔ)音情感分類問(wèn)題上,我們采用了中科院自動(dòng)化所在2016多模態(tài)情感識(shí)別競(jìng)賽中公布的數(shù)據(jù)集,共包含了生氣、焦慮、厭惡、高興、悲傷、驚訝、擔(dān)憂以及中性這八種情感類別,與數(shù)據(jù)集的基線系統(tǒng)相比,將識(shí)別率提高了4%以上,并超過(guò)了競(jìng)賽第一名所采用的一種方法。在腦運(yùn)動(dòng)想象識(shí)別分類中,我們采用BCI2008數(shù)據(jù)集,是左右手運(yùn)動(dòng)想象的二分類問(wèn)題。我們針對(duì)腦電多通道、具有空間分布特征的特點(diǎn),在C-LSTM的基礎(chǔ)上進(jìn)行了改進(jìn),將電極的空間信息通過(guò)數(shù)據(jù)整合以及小波變換腦網(wǎng)絡(luò)的方法融合其中,建立了3D-C-LSTM模型,并在識(shí)別率上相較其他方法提高了近10%,到達(dá)了92.0%,表明在腦電信號(hào)中除了時(shí)序信息之外,空間信息也是十分重要的。本文的研究工作為目前時(shí)序信號(hào)的分析處理領(lǐng)域中存在的一些關(guān)鍵性的技術(shù)問(wèn)題提供了有效的改進(jìn)方案,經(jīng)過(guò)語(yǔ)音信號(hào)和腦電信號(hào)的相關(guān)實(shí)驗(yàn)證明,CLSTM的網(wǎng)絡(luò)結(jié)構(gòu)對(duì)于時(shí)序信號(hào)的處理具有普適性,具有一定推廣價(jià)值。同時(shí)也為卷積神經(jīng)網(wǎng)絡(luò)等深度學(xué)習(xí)方法在時(shí)序信號(hào)處理中的應(yīng)用與發(fā)展提供了新的思路和方向。
[Abstract]:Time series signals are very important signals, such as speech signals, bioelectrical signals, radar and sonar signals, mechanical vibration and seismic signal [1], and so on, and so on, the amplitude of the multi component signals that are constantly changing with the change of time. The time series signal has the characteristics of nonlinear and non-stationary, and most of the current studies have been done. It is based on the assumption that the signal is short-time stationary. The feature extraction is mainly based on the frequency domain characteristics, the analysis level and the granularity are relatively simple. Moreover, the most important time sequence information in the signal is ignored, which greatly affects the ability to extract the information of the time-varying signal and limits its performance improvement in practical applications. This paper aims at this paper. The extraction and modeling of time series information in time series signals, drawing on the ability to automatically optimize and integrate multi granularity, multi time and multi-level features in the process of human brain cognition, a framework for extracting and merging multiple granularity features is proposed. We extract the characteristics of the signal according to the three granularity of frame, segment and global. The global features commonly used in the existing methods also increase the dynamic characteristics of the time series information contained in the two signals, including the frame granularity and segment granularity, effectively extracting the information in the timing signal from multiple angles, and the ability to express the information in the signal is more abundant. In the division of segment granularity, we refer to the human brain in cognitive activities. Then, we divide the length of the window into the division of the length of the window. After that, we integrate the three granularity characteristics at the frame level according to the order of time. Then we use the LSTM neural network model which has strong ability to model the time sequence information. In the concrete reality of the multi granularity characteristics, we use two methods. One is to use the method. The traditional time frequency analysis method extracts the frame features of the time series signal, and then uses the Gauss function group to convolution the frame features under the segment size window to obtain the feature of the frame. The global feature is calculated by the statistical calculation of all the frame features. The other method is combined with the breakthrough progress in various fields at present. Degree learning technology, with the help of the convolution neural network, can carry out the information extraction ability of the end to end on the original data and the feature extraction at multiple levels, to extract the multiple granularity of time sequence signals, and put forward the network structure of C-LSTM. We input the original signal to the depth convolution directly. In the network, a pre set convolution kernel is used to slide convolution on the signal, and the frame size features are obtained in the shallow CNN, while the frame granularity features are further processed with a higher level of CNN, respectively, in the medium and high level CNN output segments, as well as the global granularity. Finally, the three granularity feature information is pressed on the frame level. According to the integration of time series, the feature of multi granularity fusion is obtained, and the time series information is modeled and classified by long and short time memory network. Finally, we put forward the proposed method framework and network structure model to classify the speech emotion recognition on the speech signal and the classification and recognition of the motion imagination signal on the EEG signal. On the problem of speech emotion classification, we adopted the data set published in the more than 2016 mode emotion recognition contest of the Institute of automation of the Academy of Sciences, which included eight kinds of emotional categories, such as anger, anxiety, disgust, joy, sadness, surprise, worry and neutrality, and increased the recognition rate by more than 4% compared with the baseline system of the dataset. In the image recognition classification of brain motion, we use the BCI2008 data set in the brain motion picture recognition classification, which is the two classification problem of the left and right hand motion imagination. We improve the spatial distribution characteristics of the EEG multi-channel and have the characteristics of spatial distribution, and integrate the spatial information of the electrode through the data to integrate the data. And the method of wavelet transform brain network is fused, the 3D-C-LSTM model is established, and the recognition rate is increased by nearly 10% compared with other methods, and 92%. It shows that the spatial information is also very important in the EEG in addition to the time series information. Some key technical problems provide an effective improvement scheme. Through the experiments of speech signals and EEG signals, it is proved that the network structure of CLSTM has the universality and popularization value for the processing of time series signal. Meanwhile, it also provides the application and development of the convolution neural network and other depth learning methods in the time series signal processing. New ideas and directions are provided.
【學(xué)位授予單位】：哈爾濱工業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：R318;TP183
本文編號(hào)：2150122
[Abstract]:Time series signals are very important signals, such as speech signals, bioelectrical signals, radar and sonar signals, mechanical vibration and seismic signal [1], and so on, and so on, the amplitude of the multi component signals that are constantly changing with the change of time. The time series signal has the characteristics of nonlinear and non-stationary, and most of the current studies have been done. It is based on the assumption that the signal is short-time stationary. The feature extraction is mainly based on the frequency domain characteristics, the analysis level and the granularity are relatively simple. Moreover, the most important time sequence information in the signal is ignored, which greatly affects the ability to extract the information of the time-varying signal and limits its performance improvement in practical applications. This paper aims at this paper. The extraction and modeling of time series information in time series signals, drawing on the ability to automatically optimize and integrate multi granularity, multi time and multi-level features in the process of human brain cognition, a framework for extracting and merging multiple granularity features is proposed. We extract the characteristics of the signal according to the three granularity of frame, segment and global. The global features commonly used in the existing methods also increase the dynamic characteristics of the time series information contained in the two signals, including the frame granularity and segment granularity, effectively extracting the information in the timing signal from multiple angles, and the ability to express the information in the signal is more abundant. In the division of segment granularity, we refer to the human brain in cognitive activities. Then, we divide the length of the window into the division of the length of the window. After that, we integrate the three granularity characteristics at the frame level according to the order of time. Then we use the LSTM neural network model which has strong ability to model the time sequence information. In the concrete reality of the multi granularity characteristics, we use two methods. One is to use the method. The traditional time frequency analysis method extracts the frame features of the time series signal, and then uses the Gauss function group to convolution the frame features under the segment size window to obtain the feature of the frame. The global feature is calculated by the statistical calculation of all the frame features. The other method is combined with the breakthrough progress in various fields at present. Degree learning technology, with the help of the convolution neural network, can carry out the information extraction ability of the end to end on the original data and the feature extraction at multiple levels, to extract the multiple granularity of time sequence signals, and put forward the network structure of C-LSTM. We input the original signal to the depth convolution directly. In the network, a pre set convolution kernel is used to slide convolution on the signal, and the frame size features are obtained in the shallow CNN, while the frame granularity features are further processed with a higher level of CNN, respectively, in the medium and high level CNN output segments, as well as the global granularity. Finally, the three granularity feature information is pressed on the frame level. According to the integration of time series, the feature of multi granularity fusion is obtained, and the time series information is modeled and classified by long and short time memory network. Finally, we put forward the proposed method framework and network structure model to classify the speech emotion recognition on the speech signal and the classification and recognition of the motion imagination signal on the EEG signal. On the problem of speech emotion classification, we adopted the data set published in the more than 2016 mode emotion recognition contest of the Institute of automation of the Academy of Sciences, which included eight kinds of emotional categories, such as anger, anxiety, disgust, joy, sadness, surprise, worry and neutrality, and increased the recognition rate by more than 4% compared with the baseline system of the dataset. In the image recognition classification of brain motion, we use the BCI2008 data set in the brain motion picture recognition classification, which is the two classification problem of the left and right hand motion imagination. We improve the spatial distribution characteristics of the EEG multi-channel and have the characteristics of spatial distribution, and integrate the spatial information of the electrode through the data to integrate the data. And the method of wavelet transform brain network is fused, the 3D-C-LSTM model is established, and the recognition rate is increased by nearly 10% compared with other methods, and 92%. It shows that the spatial information is also very important in the EEG in addition to the time series information. Some key technical problems provide an effective improvement scheme. Through the experiments of speech signals and EEG signals, it is proved that the network structure of CLSTM has the universality and popularization value for the processing of time series signal. Meanwhile, it also provides the application and development of the convolution neural network and other depth learning methods in the time series signal processing. New ideas and directions are provided.
【學(xué)位授予單位】：哈爾濱工業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：R318;TP183
【參考文獻(xiàn)】
相關(guān)期刊論文 前3條
1 焦李成;楊淑媛;劉芳;王士剛;馮志璽;;神經(jīng)網(wǎng)絡(luò)七十年:回顧與展望[J];計(jì)算機(jī)學(xué)報(bào);2016年08期
2 李星雨;楊承志;曲文韜;張榮;;基于自適應(yīng)網(wǎng)格密度聚類的雷達(dá)信號(hào)分選算法[J];航天電子對(duì)抗;2013年02期
3 王登;苗奪謙;王睿智;;一種新的基于小波包分解的EEG特征抽取與識(shí)別方法研究[J];電子學(xué)報(bào);2013年01期
相關(guān)會(huì)議論文 前1條
1 賈磊;;LSTM建模和CTC訓(xùn)練在語(yǔ)音建模技術(shù)中的應(yīng)用[A];第十三屆全國(guó)人機(jī)語(yǔ)音通訊學(xué)術(shù)會(huì)議(NCMMSC2015)論文集[C];2015年
，本文編號(hào)：2150122
本文鏈接：http://sikaile.net/yixuelunwen/swyx/2150122.html
最近更新
教材專著
