本文關(guān)鍵詞： 多視點(diǎn)視頻增強(qiáng) 立體電視 幀率提升 虛擬視點(diǎn)合成 視線跟蹤　出處：《山東大學(xué)》2014年博士論文　論文類(lèi)型：學(xué)位論文

【摘要】：數(shù)字多媒體是當(dāng)前最活躍的研究領(lǐng)域之一。隨著人們對(duì)服務(wù)質(zhì)量和視覺(jué)體驗(yàn)要求的不斷提高,數(shù)字多媒體處理技術(shù)和設(shè)備都在不停的更新?lián)Q代。作為一種新興的可視媒介形式,立體電視(Three Dimensional Television,3DTV)能夠?yàn)橛脩籼峁┖軓?qiáng)的立體感和沉浸感,因而引起廣泛關(guān)注并取得一系列成果。與傳統(tǒng)二維(2D)電視相比,立體電視需要向用戶提供多個(gè)視點(diǎn)的視頻,所以傳輸數(shù)據(jù)量巨大,成為制約立體電視技術(shù)發(fā)展的瓶頸。 在立體電視系統(tǒng)中,多視點(diǎn)視頻(Multi-view Video, MVV)的空間分辨率、幀率(frame rate)、視點(diǎn)的數(shù)量等對(duì)視覺(jué)效果有著很大的影響。多視點(diǎn)視頻幀率越高,特別對(duì)于大尺寸液晶顯示設(shè)備(Liquid Crystal Display, LCD),電視畫(huà)面就會(huì)越流暢；視點(diǎn)數(shù)量越多,用戶的觀看范圍越大,立體感也就越強(qiáng)。然而,隨著傳輸視頻幀率的提高、視點(diǎn)數(shù)量的增多,傳輸?shù)臄?shù)據(jù)量必然急劇增加。雖然當(dāng)前提出了有效的多視點(diǎn)視頻編碼技術(shù),但立體電視傳輸中多視點(diǎn)視頻的幀率、視點(diǎn)數(shù)量仍然不能滿足實(shí)際要求。另外,在交互式立體電視系統(tǒng)中,通常需要對(duì)用戶視線或者頭部進(jìn)行跟蹤,依此確定視點(diǎn)的播放與切換。 針對(duì)上述立體電視系統(tǒng)中存在的問(wèn)題,本文著重對(duì)立體電視系統(tǒng)中多視點(diǎn)視頻增強(qiáng)和視線跟蹤方法進(jìn)行了研究。其中,多視點(diǎn)視頻增強(qiáng)包含立體視頻幀率提升和視點(diǎn)合成兩個(gè)方面。從發(fā)送端來(lái)看,利用幀率提升和視點(diǎn)合成可以減少數(shù)據(jù)傳輸量,節(jié)省帶寬；從接收端來(lái)進(jìn)行分析,幀率提升使單路視頻看起來(lái)更流暢,減少了由于視頻幀率過(guò)低造成的運(yùn)動(dòng)模糊和抖動(dòng),而通過(guò)視點(diǎn)合成增加了虛擬相機(jī)的數(shù)量,擴(kuò)大了可視范圍,使用戶能夠獲得更好的立體視覺(jué)體驗(yàn)。另外,本論文對(duì)交互式立體電視中的人機(jī)交互技術(shù)進(jìn)行了研究,提出了一種基于灰度分布視頻處理的非接觸式視線跟蹤系統(tǒng)。 論文在以下幾個(gè)方面的研究取得重要進(jìn)展： 1、提出了一種彩色+深度格式(video plus depth)的立體視頻幀率提升算法。首先,根據(jù)深度信息將運(yùn)動(dòng)向量(Motion Vector, MV)分成深度連續(xù)運(yùn)動(dòng)向量和深度不連續(xù)運(yùn)動(dòng)向量。在深度連續(xù)運(yùn)動(dòng)向量場(chǎng)(Motion Vector Field, MVF)中,利用深度層次約束的運(yùn)動(dòng)向量?jī)?yōu)化方法對(duì)錯(cuò)誤運(yùn)動(dòng)向量進(jìn)行檢測(cè)和校正,提高運(yùn)動(dòng)向量的準(zhǔn)確度；對(duì)于深度不連續(xù)運(yùn)動(dòng)向量場(chǎng),則采用了一種基于前景匹配的運(yùn)動(dòng)向量?jī)?yōu)化方法,保持了運(yùn)動(dòng)補(bǔ)償過(guò)程中前景運(yùn)動(dòng)物體邊緣的完整性。 2、提出了一種基于深度的適應(yīng)性運(yùn)動(dòng)補(bǔ)償和圖像塊分割方案。根據(jù)視頻場(chǎng)景中深度信息和運(yùn)動(dòng)向量的關(guān)系,適應(yīng)性選擇前向運(yùn)動(dòng)補(bǔ)償與后向運(yùn)動(dòng)補(bǔ)償,同時(shí),利用基于深度和α-matting的圖像分割方法對(duì)深度不連續(xù)圖像塊進(jìn)行分割,相對(duì)于傳統(tǒng)幀率提升算法,減少了運(yùn)動(dòng)補(bǔ)償后遮擋區(qū)域和非遮擋區(qū)域出現(xiàn)的模糊和偽影的現(xiàn)象,可以有效提升立體視頻的視覺(jué)質(zhì)量。 3、提出了一種基于非對(duì)稱圖像修復(fù)的虛擬視點(diǎn)重建算法。在提出的算法中利用左右兩路的彩色視頻和深度視頻對(duì)中間的任意虛擬視點(diǎn)的視頻進(jìn)行生成,利用虛擬視點(diǎn)與左右參考視點(diǎn)之間的空間位置關(guān)系確定主要參考視點(diǎn)和輔助參考視點(diǎn)。首先,利用三維圖像變換技術(shù)將主要參考視點(diǎn)和輔助參考視點(diǎn)圖像投影到虛擬視點(diǎn)。其次,通過(guò)圖像處理技術(shù)去除虛擬視點(diǎn)圖像中的裂紋和錯(cuò)誤點(diǎn),提高圖像質(zhì)量。然后,通過(guò)輔助虛擬視點(diǎn)圖像對(duì)主要虛擬視點(diǎn)圖像中的遮擋區(qū)域進(jìn)行填充,.并且為了實(shí)現(xiàn)視點(diǎn)間視頻色彩的統(tǒng)一,對(duì)左右參考視點(diǎn)圖像進(jìn)行亮度調(diào)整。最后,利用深度輔助的非對(duì)稱圖像修復(fù)方法對(duì)剩余的空洞區(qū)域進(jìn)行填充。 4、提出了一種基于灰度分布視頻處理的非接觸式視線跟蹤系統(tǒng),該系統(tǒng)可以作為交互式立體電視系統(tǒng)中的人機(jī)交互設(shè)備。首先,在近紅外光源照射條件下采集使用者頭動(dòng)視頻,然后根據(jù)視頻幀的灰度分布特征,依次進(jìn)行面部區(qū)域、眼睛區(qū)域、瞳孔區(qū)域的檢測(cè)與提取,最終計(jì)算出瞳孔角膜點(diǎn)反射坐標(biāo)和瞳孔中心坐標(biāo)；根據(jù)特征參數(shù),采用基于交比不變性質(zhì)的視線跟蹤算法進(jìn)行注視點(diǎn)位置的計(jì)算,實(shí)現(xiàn)視線跟蹤；另外,針對(duì)視線跟蹤過(guò)程中眼球視軸和光軸的不重合,提出了一種簡(jiǎn)單有效的五點(diǎn)定標(biāo)算法。實(shí)驗(yàn)結(jié)果表明,針對(duì)佩戴眼鏡和裸眼使用者,該系統(tǒng)均能夠達(dá)到較高的精度,能夠滿足實(shí)際需要。
[Abstract]:Digital multimedia is one of the most active research areas at present. Along with the people to the quality of service and visual experience requirements continue to increase, the digital multimedia processing technology and equipment upgrading constantly updated. As a new form of visual media, stereo TV (Three Dimensional Television, 3DTV) can provide a strong sense of three-dimensional and immersion for users, which caused widespread concern and made a series of achievements. With the traditional two-dimensional (2D) compared to the TV, stereo TV need to provide multi view video to the user, so the transmission amount of data is huge, become a bottleneck restricting the development of television technology.
In the stereo TV system, multi view video (Multi-view Video, MVV) spatial resolution, frame rate (frame rate), the number of viewpoints have great influence on visual effect. The multi view video frame rate is high, especially for large size liquid crystal display device (Liquid Crystal Display, LCD), the TV screen will be more smooth; view the more number of the user's viewing range is bigger, stereo sense is stronger. However, with the transmission of video frame rate increase, increasing the number of views, the amount of data transmission will increase rapidly. Although the proposed multiview video encoding technology, but the three-dimensional television transmission in multi view video frame rate the number of views, still can not meet the actual requirements. In addition, in the interactive television system, usually need to track the user's line of sight or head, thus confirming the playback and switching viewpoint.
In order to solve the existing problems in stereo TV system, this paper focuses on the stereo television system in multi view video enhancement and eye tracking method is studied. The multi view video enhancement synthesis contains two aspects of stereo video frame rate upgrade and viewpoint. From the sender, the frame rate up conversion and synthesis can reduce the amount of data transmission. Save bandwidth; carry on the analysis from the receiving end, the single video frame rate upgrade looks more smooth, reduce the video frame rate is too low due to motion blur and jitter, and through the virtual viewpoint synthesis increased the number of camera, expand the visual range, the user can get a better stereo vision experience. In addition, this paper the research on human-computer interaction technology for interactive stereoscopic television, this paper proposes a non-contact vision gray distribution tracking system based on video processing.
The paper has made important progress in the following aspects:
1, we propose a color plus depth format (video plus depth) stereo video frame rate upgrade algorithm. Firstly, according to the depth information of the motion vector (Motion Vector MV) is divided into continuous motion vector depth and depth discontinuity motion vectors. In continuous motion vector field (Motion Vector depth Field, MVF), using the the detection and correction of error motion vector optimization method of depth constraints, improve the accuracy of the motion vector; the depth discontinuity of motion vector field, using a motion vector optimization method, based on the prospect of the integrity of the moving foreground, motion compensation process in the edge of the object.
2, the paper proposes a segmentation scheme based on adaptive motion compensation and image block depth. According to the relationship between depth of the video scene and motion vector information, adaptive selection in forward motion compensation and backward motion compensation, at the same time, the use of image depth and alpha -matting segmentation method for discontinuous depth segmentation based on image block. Compared with the traditional frame rate upgrade algorithm, reduce the motion compensated occlusion and non occluded regions appear fuzzy and artifact phenomenon, can effectively enhance the stereoscopic video quality.
3, put forward a virtual view reconstruction algorithm based on non symmetric image restoration. In the proposed algorithm uses two color video and depth video arbitrary virtual view of the intermediate video generation, to determine the main reference view and auxiliary reference view between the virtual viewpoint and reference view about space position relations. First of all, the main reference view and auxiliary reference view image is projected to the virtual viewpoint using 3D image transform. Secondly, by removing the virtual view image of the crack and the error point of image processing technology, improve the quality of the image. Then, through the virtual view image of the occluded areas mainly in the virtual view image to fill, and in order to. To achieve unity between viewpoints about color video, reference view image brightness adjustment. Finally, the non symmetric image using depth assisted repair The residual cavity area was filled with the compound method.
4, we proposed a tracking system for non contact line of gray distribution based on video processing, the system can be used as a human-computer interaction device interactive stereoscopic television systems. First, collect the user head motion video in the near infrared light irradiation, and then according to the gray distribution features of video frames, followed by facial region, eye region detection and extraction, the pupil area, finally calculate the corneal reflection and pupil pupil coordinate center coordinates; according to the characteristic parameters, the invariance of cross ratio gaze tracking algorithm based on position calculation point of gaze, realize the eye tracking; in addition, the eye gaze tracking process of optical axis and visual axis do not coincide, put forward a five simple and effective calibration algorithm. The experimental results show that the wear glasses and naked eye users, the system can achieve high precision, can meet the The actual need.

【學(xué)位授予單位】：山東大學(xué)
【學(xué)位級(jí)別】：博士
【學(xué)位授予年份】：2014
【分類(lèi)號(hào)】：TN949.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前7條

1 溫晴川;達(dá)飛鵬;方旭;;基于雙目立體視覺(jué)的視線跟蹤系統(tǒng)標(biāo)定[J];光學(xué)學(xué)報(bào);2012年10期

2 賈茜;肖進(jìn)勝;易本順;甘良才;;基于三維遞歸搜索的多級(jí)運(yùn)動(dòng)估計(jì)視頻幀率上轉(zhuǎn)換算法[J];電子與信息學(xué)報(bào);2012年10期

3 程成;杜菁菁;藍(lán)飛翔;;眼動(dòng)交互的實(shí)時(shí)線性算法構(gòu)造和實(shí)現(xiàn)[J];電子學(xué)報(bào);2009年S1期

4 劉瑞安;靳世久;宋維;張希坤;吳曉榮;;單攝像機(jī)視線跟蹤[J];計(jì)算機(jī)應(yīng)用;2006年09期

5 楊?lèi)?ài)萍;董翠翠;侯正信;李娜;;基于多幀運(yùn)動(dòng)估計(jì)的幀率提升算法[J];計(jì)算機(jī)應(yīng)用研究;2012年10期

6 張闖;遲健男;張朝暉;高小亮;胡濤;王志良;;視線追蹤系統(tǒng)中視線估計(jì)方法研究[J];中國(guó)科學(xué):信息科學(xué);2011年05期

7 胡畔;董春鵬;吳曉榮;;基于視線跟蹤技術(shù)的眼控鼠標(biāo)[J];天津師范大學(xué)學(xué)報(bào)(自然科學(xué)版);2007年03期

，

本文編號(hào)：1494850