【摘要】：針對(duì)計(jì)算機(jī)對(duì)視頻進(jìn)行自動(dòng)標(biāo)注和描述準(zhǔn)確率不高的問(wèn)題,提出一種基于多特征融合的深度視頻自然語(yǔ)言描述的方法。該方法提取視頻幀序列的空間特征、運(yùn)動(dòng)特征、視頻特征,進(jìn)行特征的融合,使用融合的特征訓(xùn)練基于長(zhǎng)短期記憶(LSTM)的自然語(yǔ)言描述模型。通過(guò)不同的特征組合訓(xùn)練多個(gè)自然語(yǔ)言描述模型,在測(cè)試時(shí)再進(jìn)行后期融合,即先選擇一個(gè)模型獲取當(dāng)前輸入的多個(gè)可能的輸出,再使用其他模型計(jì)算當(dāng)前輸出的概率,對(duì)這些輸出的概率進(jìn)行加權(quán)求和,取概率最高的作為輸出。此方法中的特征融合的方法包括前期融合:特征的拼接、不同特征對(duì)齊加權(quán)求和;后期融合:不同特征模型輸出的概率的加權(quán)融合,使用前期融合的特征對(duì)已生成的LSTM模型進(jìn)行微調(diào)。在標(biāo)準(zhǔn)測(cè)試集MSVD上進(jìn)行實(shí)驗(yàn),結(jié)果表明:融合不同類型的特征方法能夠獲得更高評(píng)測(cè)分值的提升;相同類型的特征融合的評(píng)測(cè)結(jié)果不會(huì)高于單個(gè)特征的分值;使用特征對(duì)預(yù)訓(xùn)練好的模型進(jìn)行微調(diào)的方法效果較差。其中使用前期融合與后期融合相結(jié)合的方法生成的視頻自然語(yǔ)言描述得到的METEOR評(píng)測(cè)分值為0.302,比目前查到的最高值高1.34%,表明該方法可以提升視頻自動(dòng)描述的準(zhǔn)確性。
[Abstract]:In order to solve the problem of low accuracy of automatic video tagging and description by computer, a method of deep video natural language description based on multi-feature fusion is proposed. In this method, the spatial features, motion features, video features of video frame sequences are extracted, and the fusion features are used to train the natural language description model based on LSTM. Several natural language description models are trained by different feature combinations, and later fusion is carried out during the test. That is to say, one model is selected to obtain multiple possible outputs of the current input first, and then the probability of the current output is calculated by using other models. The probability of these outputs is weighted and the highest probability is taken as the output. The methods of feature fusion in this method include early fusion: feature splicing, weighted summation of different feature alignment, and later fusion: weighted fusion of probability of output of different feature models. The pre-fusion features are used to fine-tune the generated LSTM model. The experiment on MSVD standard test set shows that the fusion of different types of features can obtain higher evaluation scores, the same type of feature fusion results are not higher than the single feature scores, and the results of the same type of feature fusion are not higher than that of the single feature, and the evaluation results of the same type of feature fusion are not higher than that of the single feature. The method of fine-tuning the pre-trained model by using features is not effective. Among them, the METEOR value of the video natural language description generated by the combination of pre-fusion and post-fusion is 0.302, which is 1.34 higher than the highest value found at present, which indicates that this method can improve the accuracy of video automatic description.
【作者單位】： 電子科技大學(xué)信息與軟件工程學(xué)院;電子科技大學(xué)計(jì)算機(jī)科學(xué)與工程學(xué)院;
【基金】：國(guó)家自然科學(xué)基金資助項(xiàng)目(61300192) 中央高�；究蒲袠I(yè)務(wù)費(fèi)專項(xiàng)資金資助項(xiàng)目(ZYGX2014J052)~~
【分類號(hào)】：TP391.41

【相似文獻(xiàn)】

相關(guān)期刊論文 前4條

1 張珂;王小捷;鐘義信;;在觀察者參考框架下的空間關(guān)系自然語(yǔ)言描述[J];清華大學(xué)學(xué)報(bào)(自然科學(xué)版);2011年12期

2 張珂;鐘義信;王小捷;靳越;;在外部參考框架下的空間關(guān)系自然語(yǔ)言描述[J];北京郵電大學(xué)學(xué)報(bào);2012年03期

3 應(yīng)申;李霖;高玉榮;甘莉;姜文亮;余建偉;;LBS中多尺度空間方位的自然語(yǔ)言描述[J];地理信息世界;2008年01期

4 ;[J];;年期

相關(guān)碩士學(xué)位論文 前2條

1 袁錄;基于F直方圖的空間位置關(guān)系自然語(yǔ)言描述自動(dòng)生成研究[D];北京郵電大學(xué);2012年

2 王霄;自然語(yǔ)言描述的空間實(shí)體自動(dòng)擺放技術(shù)研究[D];哈爾濱工業(yè)大學(xué);2014年

，

本文編號(hào)：2123771