本文選題：多視點(diǎn) + 深度圖�。� 參考：《華中科技大學(xué)》2016年博士論文

【摘要】：隨著視頻技術(shù)的發(fā)展,能夠帶來沉浸式體驗(yàn)的3D視頻(3D video,3DV)和能夠?qū)崿F(xiàn)觀眾與內(nèi)容商之間交互的自由視點(diǎn)視頻(free viewpoint video, FVV)引起了高度的重視和廣泛的研究。此類新型視頻系統(tǒng)中,利用多視點(diǎn)深度信息不但可以大大減少所需的視頻數(shù)據(jù)量,還可實(shí)現(xiàn)任意視點(diǎn)繪制,提高系統(tǒng)的靈活性；因此,多視點(diǎn)深度信息具有十分基礎(chǔ)又至關(guān)重要的作用。另一方面,深度值表征場景到成像平面的距離,它只能通過測量或者計(jì)算的方法獲得,其中必然引入錯(cuò)誤。因此,高質(zhì)量深度圖的獲取及其質(zhì)量評(píng)估成為3D視頻和自由視點(diǎn)視頻系統(tǒng)中意義重大的問題。本文圍繞該課題展開了三個(gè)方面的研究。首先,本文研究了多臺(tái)RGB-D相機(jī)對(duì)場景進(jìn)行多視點(diǎn)深度圖采集的方案,解決了多臺(tái)深度相機(jī)之間干擾的問題；然后,考慮到現(xiàn)有RGB-D相機(jī)產(chǎn)品無法滿足新的視頻應(yīng)用對(duì)深度圖的需求,本文提出了一種基于相位結(jié)構(gòu)光和主動(dòng)立體匹配的混合式采集方案以獲取深度圖；最后,本文研究了深度圖的質(zhì)量評(píng)估問題,考慮到無失真的參考深度圖往往不存在,本文提出了一種基于其物理意義的無參考方案實(shí)現(xiàn)深度值的錯(cuò)誤檢測和質(zhì)量評(píng)估。首先,深度相機(jī)技術(shù)在最近幾年發(fā)展迅速,以微軟Kinect為代表的RGB-D相機(jī)可快捷地獲取深度圖。然而,RGB-D相機(jī)并沒有多設(shè)備協(xié)同機(jī)制,當(dāng)多臺(tái)RGB-D相機(jī)采集同一場景的深度時(shí)會(huì)發(fā)生干擾,導(dǎo)致深度圖質(zhì)量下降。本文深入理解了干擾機(jī)制,分析了干擾的影響,進(jìn)而提出了一種消除干擾以恢復(fù)深度值的方案。分析表明,RGB-D相機(jī)具有一定的魯棒性,使干擾引起深度值丟失而非深度值變化,故可用有效深度值恢復(fù)出丟失的深度信息。另外,考慮深度圖在物體內(nèi)部和邊緣的性質(zhì)差異,本文進(jìn)一步提出了區(qū)域自適應(yīng)的深度值恢復(fù)方案。該方案首先以紋理圖為參考,將深度圖劃分為平滑干擾區(qū)域和邊界干擾區(qū)域。對(duì)平滑干擾區(qū)域,在梯度域建立馬爾科夫隨機(jī)場(Markov random field, MRF)恢復(fù)出梯度,進(jìn)而利用離散泊松方程(discrete Poisson equation, DPE)恢復(fù)深度值。對(duì)邊界區(qū)域,在深度值空間內(nèi)建立紋理引導(dǎo)的MRF模型求取深度值。該方案在保留物體內(nèi)部平滑性的同時(shí),也維持了物體之間尖銳的邊緣,保留了場景的幾何信息。其次,3DV和FVV等新的視頻應(yīng)用對(duì)深度圖提出了較高的要求。高質(zhì)量深度圖需要精確、稠密,通過單幀即可獲取以便適用于動(dòng)態(tài)場景,并可擴(kuò)展到多視點(diǎn)深度圖采集；而現(xiàn)有的RGB-D相機(jī)并不能良好地滿足該需求。因此,本文以現(xiàn)有結(jié)構(gòu)光測距方法為基礎(chǔ),提出了一種混合方案來獲取高質(zhì)量深度圖。該方案提出一種基于條帶的多頻率正弦波模板,該模板具有正弦波模板可攜帶相位信息的特點(diǎn),還具有良好的局部唯一性,從而使深度圖可通過混合方案來獲取。具體地,每個(gè)解碼條帶內(nèi)的模板為正弦波,可利用傅里葉變換輪廓測定法(Fourier transform profilimetry, FTP)計(jì)算包裹相位,然后基于深度圖的局部平滑性和解碼條帶之間多頻率的性質(zhì)準(zhǔn)確而快速地進(jìn)行相位展開,并轉(zhuǎn)化成視差和深度值。此外,對(duì)不滿足平滑性的區(qū)域,本文進(jìn)一步利用模板的局部唯一性,通過空域立體匹配修正其深度值。實(shí)驗(yàn)結(jié)果表明,本文提出的方案對(duì)深度值跳變和空間孤立物體等復(fù)雜場景均能準(zhǔn)確獲得深度值,并可結(jié)合復(fù)用技術(shù)獲取多視點(diǎn)下的深度圖。然后,深度圖通過測量和計(jì)算獲得,其中錯(cuò)誤難以避免,需要進(jìn)行檢測；另一方面,無失真的參考深度圖并不存在,所以常見的全參考和部分參考的質(zhì)量評(píng)估途徑對(duì)深度圖并不適用。針對(duì)該問題,本文提出了一種無參考的深度圖質(zhì)量評(píng)估方案。該方案以深度圖的物理意義為理論依據(jù),重點(diǎn)考察深度圖邊緣的幾何扭曲�？紤]到參考圖像缺失,該方案從紋理圖和深度圖的相關(guān)性出發(fā),對(duì)深度邊緣和紋理邊緣進(jìn)行匹配。具體來說,紋理圖和深度圖是場景的兩種表現(xiàn)形式,故二者的邊緣具有很強(qiáng)的相關(guān)性。本文采用了邊緣的空間位置、方向和長度為特征建立邊緣的相似性度量,并以此實(shí)現(xiàn)邊緣匹配；同時(shí)方案還采用了基于邊緣線段的匹配方法以提高魯棒性。最后,基于兩類邊緣線段之間匹配的結(jié)果,可確定深度圖中的壞點(diǎn)并量化深度圖質(zhì)量。實(shí)驗(yàn)結(jié)果表明,該方案能準(zhǔn)確檢測到深度圖的邊緣失真并確定壞點(diǎn)；實(shí)驗(yàn)表明方案中提出的無參考質(zhì)量指標(biāo)和現(xiàn)有的全參考指標(biāo)高度相關(guān),同時(shí)也和虛擬視點(diǎn)的質(zhì)量緊密相關(guān)。該錯(cuò)誤檢測方案還可用于深度錯(cuò)誤校正和質(zhì)量增強(qiáng)等后續(xù)工作。最后,本文對(duì)以上研究內(nèi)容和創(chuàng)新點(diǎn)進(jìn)行了總結(jié),并結(jié)合視頻技術(shù)發(fā)展的趨勢,展望了本文的后續(xù)工作。本文工作是對(duì)多視點(diǎn)深度圖采集和無參考條件下深度圖質(zhì)量評(píng)估的探索,為3DV和FVV等以深度圖為基礎(chǔ)的視頻系統(tǒng)應(yīng)用提供了研究思路和方案。
[Abstract]:With the development of video technology, the 3D video (3D video, 3DV) and the free viewpoint video (free viewpoint video, FVV) that can interact with the audience and the content merchants (FVV) have aroused great attention and extensive research. In this kind of new video system, the use of multi view depth information can not only greatly reduce the depth of the video. The amount of video data required can also be used to draw any view points and improve the flexibility of the system. Therefore, the depth information of multi view points has a very basic and vital role. On the other hand, the depth values can only be obtained by measuring or calculating the distance from the scene to the imaging plane, in which the errors are inevitably introduced. Therefore, the high quality is high quality. The acquisition of the quantity depth map and its quality evaluation have become a significant problem in the 3D video and the free view video system. This paper focuses on three aspects of this topic. Firstly, this paper studies the multi view depth map acquisition scheme of multiple RGB-D cameras for the scene, and solves the problem of interference between many deep cameras. In view of the fact that the existing RGB-D camera products are unable to meet the needs of the new video application to the depth map, a hybrid acquisition scheme based on the phase structure light and active stereo matching is proposed to obtain the depth map. Finally, the quality evaluation of the depth map is studied in this paper, considering that the undistorted reference depth map is often not stored. In this paper, a kind of error detection and quality evaluation of depth value based on its physical meaning is proposed. First, the depth camera technology has developed rapidly in recent years. The depth map can be quickly obtained by the RGB-D camera represented by Microsoft Kinect. However, the RGB-D camera has no multi device synergy mechanism, when multiple RGB-D cameras are used. When collecting the depth of the same scene, interference will occur and the quality of the depth map is reduced. This paper deeply understands the interference mechanism, analyzes the influence of interference, and then proposes a scheme to eliminate the interference to restore the depth value. The analysis shows that the RGB-D camera has a certain robustness, causing the interference to cause the depth value to be lost rather than the depth value. In addition, considering the difference in the nature of the interior and edge of the object, the depth recovery scheme of the region adaptive is further proposed in this paper. This scheme first uses the texture map as a reference to divide the depth map into a smooth interference region and a boundary interference region. The gradient is restored in the gradient domain (Markov random field, MRF), and then the depth value is restored by the discrete Poisson equation (discrete Poisson equation, DPE). The depth value of the texture guided MRF model is established in the boundary area in the depth value space. The scheme is also maintained at the same time, while preserving the interior smoothness of the object. The sharp edges between objects are held and the geometric information of the scene is retained. Secondly, new video applications such as 3DV and FVV have higher requirements for the depth map. The high quality depth map needs accurate and dense, can be obtained by single frame to apply to dynamic scenes and can be extended to multi view depth map collection; and the existing RGB-D cameras are Therefore, in this paper, a hybrid scheme is proposed to obtain a high quality depth map based on the existing structured light ranging method. A multi frequency sine wave template based on strip is proposed. The template has a sinusoidal template with the characteristics of carrying phase information, and has good local uniqueness. In particular, the template within each decoding strip is a sine wave, and the Fourier transform profilimetry (FTP) can be used to calculate the phase of the wrapping, then the local smoothness of the depth map and the properties of the multi frequency between the decoded bands are accurate and fast. In addition, in the area where the smoothness is not satisfied, this paper further uses the local uniqueness of the template to amend the depth value by spatial stereo matching. The experimental results show that the proposed scheme can obtain the depth value accurately for the complex scenes such as the depth jump and the space isolated body and so on. And then, the depth map is obtained under multi view points. Then, the depth map is obtained by measurement and calculation, in which the errors are difficult to avoid and need to be detected; on the other hand, the undistorted reference depth map does not exist, so the common reference and partial reference quality assessment approach does not apply to the depth map. This scheme is based on the physical meaning of the depth map, focusing on the geometric distortion of the edge of the depth map. Considering the lack of reference images, the scheme matches the correlation of the texture map and the depth map to the depth edge and the fringe edge. Specifically, texture and depth. The degree map is the two representation of the scene, so the edge of the two has a strong correlation. In this paper, the edge location, direction and length are used to establish the similarity measure of the edge, and the edge matching is realized. At the same time, the scheme also uses the matching method based on the edge line segment to improve the robustness. Finally, the two class is based on the two classes. The result of the matching between the edge lines can determine the bad points in the depth map and quantify the quality of the depth map. The experimental results show that the scheme can accurately detect the edge distortion of the depth map and determine the bad point; the experiment shows that the non reference quality index proposed in the scheme is highly correlated with the existing full reference index, and also the quality of the virtual view. The error detection scheme can also be used in the follow-up work of depth error correction and quality enhancement. Finally, this paper summarizes the above research content and innovation points, and combines the trend of video technology development, and looks forward to the follow-up work of this paper. This work is the acquisition of multi view depth map and the depth of no reference conditions. The exploration of graph quality assessment provides research ideas and solutions for the application of depth map based video systems such as 3DV and FVV.

【學(xué)位授予單位】：華中科技大學(xué)
【學(xué)位級(jí)別】：博士
【學(xué)位授予年份】：2016
【分類號(hào)】：TP391.41

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 左一帆;安平;張兆楊;;基于圖割的高質(zhì)量深度圖獲取方法[J];電視技術(shù);2011年15期

2 張艷;安平;張秋聞;王奎;張兆楊;;恰可察覺深度差異模型的深度圖優(yōu)化方法[J];光電子.激光;2012年07期

3 左宇鵬;;基于深度圖壓縮的邊界自適應(yīng)上采樣方案[J];計(jì)算機(jī)與現(xiàn)代化;2014年05期

4 楊超;安平;何宛文;王健鑫;張兆楊;;一種用于深度圖編碼的虛擬視失真估計(jì)模型[J];光電子.激光;2014年07期

5 溫宇強(qiáng),李德華;多視角深度圖融合方法綜述[J];計(jì)算機(jī)與數(shù)字工程;2003年04期

6 葉長明;蔣建國;詹曙;S.Ando;;不同姿態(tài)人臉深度圖識(shí)別的研究[J];電子測量與儀器學(xué)報(bào);2011年10期

7 左一帆;安平;馬然;沈禮權(quán);張兆楊;;深度圖時(shí)域一致性增強(qiáng)[J];光電子.激光;2014年01期

8 朱波;蔣剛毅;張?jiān)?郁梅;;面向虛擬視點(diǎn)圖像繪制的深度圖編碼算法[J];光電子.激光;2010年05期

9 周娟;李勇平;黃躍峰;;基于強(qiáng)度圖和深度圖的多模態(tài)人臉識(shí)別[J];計(jì)算機(jī)工程與應(yīng)用;2012年25期

10 李思宇;李勝;王衡;汪國平;;利用深度圖優(yōu)化的自然光束體構(gòu)建與繪制[J];計(jì)算機(jī)輔助設(shè)計(jì)與圖形學(xué)學(xué)報(bào);2011年07期

相關(guān)會(huì)議論文 前3條

1 陳東;楊生鵬;莊嚴(yán);王偉;;基于視覺信息的三維激光點(diǎn)云渲染與深度圖構(gòu)建[A];第二十九屆中國控制會(huì)議論文集[C];2010年

2 劉偉鋒;張卓;王延江;;基于光線衰減的深度獲取方法[A];中國自動(dòng)化學(xué)會(huì)控制理論專業(yè)委員會(huì)B卷[C];2011年

3 張帥;付宏杰;;基于Kinect的多點(diǎn)觸控系統(tǒng)研究與實(shí)現(xiàn)[A];第六屆全國信號(hào)和智能信息處理與應(yīng)用學(xué)術(shù)會(huì)議論文集[C];2012年

相關(guān)博士學(xué)位論文 前10條

1 李賀建;三維視頻中基于FPGA的實(shí)時(shí)深度估計(jì)研究與應(yīng)用[D];上海大學(xué);2015年

2 馬祥;提高三維視頻深度編碼性能的技術(shù)研究[D];西安電子科技大學(xué);2015年

3 葛川;三維視頻的高效壓縮及資源分配算法研究[D];山東大學(xué);2015年

4 葉昕辰;面向3DTV的深度計(jì)算重建[D];天津大學(xué);2015年

5 王來花;基于深度的虛擬視點(diǎn)繪制及其失真研究[D];天津大學(xué);2015年

6 郭莉琳;基于3D-HEVC的三維視頻編碼快速算法研究[D];浙江大學(xué);2017年

7 向森;多視點(diǎn)深度圖采集與質(zhì)量評(píng)估方法研究[D];華中科技大學(xué);2016年

8 鄧慧萍;3D視頻的深度圖優(yōu)化與深度編碼方法研究[D];華中科技大學(xué);2013年

9 高凱;立體視頻深度圖提取及深度序列編碼技術(shù)研究[D];吉林大學(xué);2013年

10 羅雷;基于深度圖繪制的三維視頻編碼技術(shù)研究[D];浙江大學(xué);2013年

相關(guān)碩士學(xué)位論文 前10條

1 李海坤;基于彩色和深度的前景分割研究[D];山東大學(xué);2015年

2 張?jiān)罋g;3D視頻編碼中深度信息優(yōu)化及場景背景編碼技術(shù)研究[D];哈爾濱工業(yè)大學(xué);2015年

3 曹廣昊;立體視頻系統(tǒng)中深度傳播算法的研究[D];山東大學(xué);2015年

4 丁焱;基于深度圖的虛擬視點(diǎn)繪制中空洞填補(bǔ)技術(shù)研究[D];哈爾濱工業(yè)大學(xué);2015年

5 馬姝穎;基于視點(diǎn)合成的深度圖編碼技術(shù)研究[D];電子科技大學(xué);2015年

6 王旭;面向繪制質(zhì)量的深度圖壓縮感知研究[D];上海大學(xué);2015年

7 卞玲艷;基于深度圖的2D轉(zhuǎn)3D視頻算法的研究[D];電子科技大學(xué);2015年

8 王亞峰;3D-HEVC深度圖預(yù)處理與誤碼掩蓋技術(shù)研究[D];西安電子科技大學(xué);2014年

9 白樹斌;基于RGB-D圖像的深度圖增強(qiáng)問題研究[D];青島大學(xué);2015年

10 王玉峰;基于多視點(diǎn)的三維場景的低延遲遠(yuǎn)程繪制算法研究[D];浙江工商大學(xué);2015年

，

本文編號(hào)：1848559