本文關(guān)鍵詞： 人臉檢測 表情識(shí)別 語音情感識(shí)別 多模態(tài)融合　出處：《江蘇大學(xué)》2017年碩士論文　論文類型：學(xué)位論文

【摘要】：駕駛員路怒癥目前已是影響安全駕駛的一個(gè)很重要因素,它是由于交通阻塞情況下開車壓力與挫折引起的駕駛員憤怒的情緒�！奥放Y”駕駛員會(huì)襲擊他人的汽車,惡意違反交通規(guī)則,引發(fā)交通事故。路怒癥自動(dòng)檢測與預(yù)警技術(shù)的研究已成為主動(dòng)安全駕駛技術(shù)的重要組成部分。近年來駕駛員路怒癥研究得到廣泛關(guān)注,但大部分的研究主要集中在從心理學(xué)、政策、法規(guī)方面如何避免怒路癥的發(fā)生,而針對路怒癥自動(dòng)檢測和識(shí)別技術(shù)的研究還比較少。情感識(shí)別領(lǐng)域的研究表明,人的表情和語音是表現(xiàn)情感的兩個(gè)重要通道。因此,本文在詳細(xì)分析國內(nèi)外表情識(shí)別和語音情感識(shí)別以及駕駛員路怒癥檢測技術(shù)最新進(jìn)展的基礎(chǔ)上,結(jié)合Kinect設(shè)備所采集的紅外、深度信息和語音信息,研究在駕駛條件下駕駛員人臉檢測、路怒表情識(shí)別、路怒語音情感識(shí)別的問題,并提出融合表情和語音的駕駛員路怒癥識(shí)別方法,最后通過實(shí)驗(yàn)進(jìn)行驗(yàn)證。本文主要工作如下:(1)錄制Kinect駕駛員路怒行為數(shù)據(jù)庫。鑒于目前國內(nèi)外沒有基于Kinect較為完備的駕駛員路怒行為數(shù)據(jù)庫,課題組組織并錄制了包含駕駛員Infrared-D(紅外和深度)信息、駕駛員面部表情Infrared-D信息、駕駛員情感語音數(shù)據(jù)庫。(2)提出融合Infrared-D信息的駕駛員人臉檢測方法。該方法首先通過紅外和深度信息的融合得到圖像中的駕駛員區(qū)域;然后,采用卷積網(wǎng)絡(luò)人臉檢測器遍歷駕駛員區(qū)域圖像得到駕駛員人臉的可能位置;進(jìn)而使用級聯(lián)的卷積網(wǎng)絡(luò)人臉檢測器進(jìn)一步縮小駕駛員人臉定位區(qū)域;最后,使用NMS(Non-maximum suppression)得到駕駛員人臉最終窗口。該方法和多種現(xiàn)有的方法比較,取得較好的結(jié)果,在準(zhǔn)確率和召回率平均達(dá)到97.3%和84.4%。(3)基于PCANet,提出一種融合面部Infrared-D圖像的駕駛員路怒表情識(shí)別方法。該方法首先使用駕駛員面部的紅外圖像和深度圖像訓(xùn)練PCANet過濾器,提取面部紅外圖像和深度圖像的特征圖,再對得到的特征圖分別進(jìn)行哈希編碼,進(jìn)而對得到的哈希編碼圖采用疊加操作進(jìn)行融合,并對融合后的特征圖提取直方圖特征作為最后的情感特征;最后,采用所提取的情感特征訓(xùn)練SVM,進(jìn)行駕駛員路怒表情和非路怒表情的識(shí)別。該駕駛員路怒表情識(shí)別方法的有效性在實(shí)驗(yàn)中得到驗(yàn)證,其準(zhǔn)確率達(dá)到74.6%。(4)提出融合面部表情和語音信號(hào)的駕駛員路怒癥識(shí)別方法。該方法首先采用多任務(wù)卷積神經(jīng)網(wǎng)絡(luò)從聲音信號(hào)和說話內(nèi)容兩個(gè)方面識(shí)別駕駛員語音情感,然后判斷駕駛員是否說話,如果不說話則將駕駛員表情識(shí)別的結(jié)果作為駕駛員路怒癥檢測的結(jié)果;如果說話,則將語音情感識(shí)別的結(jié)果作為駕駛員路怒癥檢測的結(jié)果;最后,對30s內(nèi)的駕駛員表情和語音情感識(shí)別結(jié)果進(jìn)行投票,投票最多的作為最終駕駛員路怒癥的識(shí)別結(jié)果。
[Abstract]:Driver road rage is now an important factor affecting safe driving. It is a result of driver anger caused by driving stress and frustration in traffic jams. Road rage drivers attack other people's cars. The research on automatic detection and early warning of road rage has become an important part of active safe driving technology. In recent years, the study of driver road rage has received extensive attention. But most of the research focuses on how to avoid road rage in psychology, policy and regulation, but there are few researches on automatic detection and recognition of road rage. Human expression and speech are two important ways to express emotion. Therefore, based on the detailed analysis of the latest development of expression recognition, speech emotion recognition and driver road rage detection technology at home and abroad, this paper combines the infrared data collected by Kinect equipment. In this paper, the problems of driver's face detection, road rage expression recognition and road rage speech emotion recognition under driving condition are studied, and the identification method of driver's road rage disease by combining facial expression and speech is put forward. The main work of this paper is as follows: 1) record Kinect driver's road rage behavior database. In view of the fact that there is no complete driver road rage behavior database based on Kinect at home and abroad, The team organized and recorded the driver infrared-D (infrared and depth) information, the driver's facial expression Infrared-D information, Driver affective voice database. (2) A driver face detection method based on Infrared-D information is proposed. Firstly, the driver region in the image is obtained by the fusion of infrared and depth information. Using convolution network face detector to traverse the driver's region image to get the possible position of driver's face; then using cascaded convolution network face detector to further reduce the driver's face location area; finally, The final window of driver's face is obtained by NMS(Non-maximum expression. Compared with many existing methods, this method has good results. Based on PCANet, a road rage recognition method based on facial Infrared-D images is proposed. Firstly, the infrared and depth images of driver's face are used to train the PCANet filter. The feature images of facial infrared images and depth images are extracted, and the obtained feature images are hashing respectively, and the resulting hash coding images are fused by superposition operation. Finally, the histogram feature is extracted as the final affective feature. SVM was trained with the extracted emotional features to recognize the driver's road anger expression and non-road anger expression, and the effectiveness of the method was verified in the experiment. The method of identification of driver's road rage is put forward, which combines facial expression and speech signal. Firstly, the multi-task convolution neural network is used to recognize the driver's speech emotion from two aspects: sound signal and speech content. Then the driver is judged whether to speak, if he does not speak, the result of driver's facial expression recognition is regarded as the result of driver's road rage detection; if he speaks, the result of speech emotion recognition is regarded as the result of driver's road rage detection; finally, the result of speech emotion recognition is regarded as the result of driver's road rage detection. The results of driver's facial expression and speech emotion recognition were voted in 30s, and the result of the final driver's road rage recognition was the most.
【學(xué)位授予單位】：江蘇大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TN912.34;TP391.41

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 趙書輝;李蘭蘭;付秀華;;教師的面部表情探析[J];科教文匯(下旬刊);2009年10期

2 ;日本研制美女機(jī)器人演員:有65種面部表情[J];科技傳播;2012年08期

3 陳瀟;;不同情緒狀態(tài)對于面部表情判斷的影響[J];大眾商務(wù);2009年22期

4 李娟;;讀懂你的表情——保羅·艾科曼的面部表情研究[J];知識(shí)就是力量;2009年12期

5 王展;;喜怒無常的面部表情——卡通面部表情練習(xí)[J];中國校外教育(美術(shù));2010年04期

6 徐超;馮志勇;;基于協(xié)同交互的面部表情實(shí)時(shí)分析[J];南開大學(xué)學(xué)報(bào)(自然科學(xué)版);2010年03期

7 陸橋達(dá);;跨文化交際中的面部表情交流簡析(英文)[J];商品與質(zhì)量;2010年SC期

8 ;最新紅外線傳感儀 可通過面部表情控制汽車[J];金卡工程;2012年07期

9 ;日本制成有面部表情的機(jī)器人[J];機(jī)器人情報(bào);1994年02期

10 張海波;面部表情的仿真研究[J];計(jì)算機(jī)仿真;1999年02期

相關(guān)會(huì)議論文 前10條

1 張松林;張忠秋;;世界頂級男子跳水三米板運(yùn)動(dòng)員奧運(yùn)會(huì)決賽面部表情特征研究[A];第九屆全國體育科學(xué)大會(huì)論文摘要匯編(2)[C];2011年

2 王翔南;;人類表情與情緒的相關(guān)性探討[A];中國心理衛(wèi)生協(xié)會(huì)第四屆學(xué)術(shù)大會(huì)論文匯編[C];2003年

3 王振宏;田博;崔雪融;;3-6歲幼兒面部表情指認(rèn)與命名能力的發(fā)展特點(diǎn)[A];第十一屆全國心理學(xué)學(xué)術(shù)會(huì)議論文摘要集[C];2007年

4 孟昭蘭;閻軍;孟憲東;;中國兒童面部表情模式制作及分析[A];全國第五屆心理學(xué)學(xué)術(shù)會(huì)議文摘選集[C];1984年

5 張竇斐;孫丹;李文輝;蔣重清;;快速反應(yīng)條件下動(dòng)態(tài)呈現(xiàn)阻礙面部表情加工的腦電研究[A];增強(qiáng)心理學(xué)服務(wù)社會(huì)的意識(shí)和功能——中國心理學(xué)會(huì)成立90周年紀(jì)念大會(huì)暨第十四屆全國心理學(xué)學(xué)術(shù)會(huì)議論文摘要集[C];2011年

6 嚴(yán)t榯，

本文編號(hào)：1527489