基于FPGA的實(shí)時(shí)雙目立體視覺(jué)系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)
發(fā)布時(shí)間：2019-05-10 20:39
【摘要】：立體視覺(jué)技術(shù)是計(jì)算機(jī)視覺(jué)領(lǐng)域中非常重要的研究方向,其通過(guò)匹配一對(duì)不同視角的圖像來(lái)提取現(xiàn)實(shí)場(chǎng)景中的深度信息,該技術(shù)在無(wú)人駕駛、無(wú)人機(jī)、虛擬現(xiàn)實(shí)、人機(jī)交互以及3DTV等領(lǐng)域廣泛應(yīng)用。在過(guò)去幾十年中,提出了許多算法以及設(shè)計(jì)平臺(tái)來(lái)提高系統(tǒng)的精確度與實(shí)時(shí)性,然而由于在獲取立體視覺(jué)的深度信息時(shí),其運(yùn)算量巨大且算法復(fù)雜度較高,所以實(shí)時(shí)獲得高質(zhì)量的深度信息仍然具有較大的挑戰(zhàn)性。本文主要提出了基于FPGA+ARM架構(gòu)的實(shí)時(shí)雙目立體視覺(jué)的軟、硬件系統(tǒng)設(shè)計(jì)方案。根據(jù)系統(tǒng)方案要求,系統(tǒng)主要包括圖像傳感器視頻的高速接收、極線幾何校正、Census變換、半全局立體匹配算法、左右一致性校驗(yàn)以及中值濾波等關(guān)鍵模塊。整個(gè)立體視覺(jué)的系統(tǒng)在單一的Xilinx ZC706開(kāi)發(fā)板上實(shí)現(xiàn),硬件平臺(tái)以XC7Z045芯片為處理器核心,根據(jù)對(duì)系統(tǒng)軟、硬件功能劃分,PS(ARM)端主要是實(shí)現(xiàn)軟件控制與算法的調(diào)試,PL(FPGA)端主要實(shí)現(xiàn)視覺(jué)算法的并行處理的硬件加速。根據(jù)系統(tǒng)支持不同的分辨率輸出以及算法的配置模式,系統(tǒng)采用了模塊化、參數(shù)化的設(shè)計(jì)思想。在系統(tǒng)設(shè)計(jì)過(guò)程中,攝像頭采用APTINA的MT9V034,接口數(shù)據(jù)為高速LVDS串行傳輸,可以減少噪聲等干擾;極線幾何校正采用二元回歸多項(xiàng)式方程來(lái)模擬矩陣相乘,減少兩幅圖像坐標(biāo)存儲(chǔ),大大減少了片上硬件邏輯資源。多項(xiàng)式計(jì)算采用流水線技術(shù),明顯提高系統(tǒng)的工作頻率;由于半全局立體匹配算法的復(fù)雜度以及對(duì)存儲(chǔ)器的要求很高,本文采用有效的計(jì)算代價(jià)函數(shù)的方法,并簡(jiǎn)化為4個(gè)方向進(jìn)行代價(jià)聚合,設(shè)計(jì)并行緩存器,降低對(duì)存儲(chǔ)器的要求。中值濾波可以濾除誤匹配點(diǎn)以及動(dòng)態(tài)規(guī)劃帶來(lái)的條紋等問(wèn)題。實(shí)驗(yàn)結(jié)果表明,本設(shè)計(jì)的系統(tǒng)可以達(dá)到視頻的實(shí)時(shí)處理要求,圖像的分辨率為640×480,最大視差搜索范圍為64個(gè)像素,幀率60 fps。對(duì)于匹配算法模塊的工作頻率可以達(dá)到130 MHz,可以實(shí)現(xiàn)1280×1024@72.2 fps的實(shí)時(shí)視頻處理。
[Abstract]:Stereo vision technology is a very important research direction in the field of computer vision. It extracts depth information from real scene by matching a pair of images with different angles of view. This technology is used in unmanned, unmanned aerial vehicles, virtual reality, and so on. Human-computer interaction and 3DTV are widely used. In the past decades, many algorithms and design platforms have been proposed to improve the accuracy and real-time performance of the system. However, because of the huge computational complexity and the high complexity of the algorithm, the depth information of stereo vision is obtained. Therefore, it is still challenging to obtain high quality depth information in real time. In this paper, the software and hardware design scheme of real-time binocular stereo vision based on FPGA ARM architecture is proposed. According to the requirements of the system scheme, the system mainly includes high-speed video reception of image sensor, polar geometric correction, Census transform, semi-global stereo matching algorithm, left and right consistency check and median filtering and other key modules. The whole stereo vision system is implemented on a single Xilinx ZC706 development board. The hardware platform is based on the XC7Z045 chip as the processor core. According to the software and hardware functions of the system, the, PS (ARM) terminal is divided into two parts: software control and algorithm debugging. The PL (FPGA) side mainly realizes the hardware acceleration of parallel processing of visual algorithm. According to the different resolution output and the configuration mode of the algorithm, the system adopts the design idea of modularization and parametrization. In the process of system design, the camera uses APTINA MT9V034, interface data for high-speed LVDS serial transmission, which can reduce noise and other interference. The polar line geometry correction uses the binary regression polynomial equation to simulate the matrix multiplication, reduces the coordinate storage of two images, and greatly reduces the on-chip hardware logic resources. The pipeline technology is used to calculate the multinomial, which obviously improves the working frequency of the system. Because of the complexity of semi-global stereo matching algorithm and the high requirement for memory, this paper adopts an effective method of computing cost function, and simplifies it to four directions for cost aggregation, designs parallel buffer, and reduces the requirement of memory. Median filtering can filter out mismatching points and stripes caused by dynamic programming. The experimental results show that the system can meet the requirements of real-time video processing. The resolution of the image is 640x480, the maximum disparity search range is 64 pixels, and the frame rate is 60 fps.. For the matching algorithm module, the working frequency of the matching algorithm module can reach 130 MHz, and the real-time video processing of 1280 脳 104 鈮，
本文編號(hào)：2473972
[Abstract]:Stereo vision technology is a very important research direction in the field of computer vision. It extracts depth information from real scene by matching a pair of images with different angles of view. This technology is used in unmanned, unmanned aerial vehicles, virtual reality, and so on. Human-computer interaction and 3DTV are widely used. In the past decades, many algorithms and design platforms have been proposed to improve the accuracy and real-time performance of the system. However, because of the huge computational complexity and the high complexity of the algorithm, the depth information of stereo vision is obtained. Therefore, it is still challenging to obtain high quality depth information in real time. In this paper, the software and hardware design scheme of real-time binocular stereo vision based on FPGA ARM architecture is proposed. According to the requirements of the system scheme, the system mainly includes high-speed video reception of image sensor, polar geometric correction, Census transform, semi-global stereo matching algorithm, left and right consistency check and median filtering and other key modules. The whole stereo vision system is implemented on a single Xilinx ZC706 development board. The hardware platform is based on the XC7Z045 chip as the processor core. According to the software and hardware functions of the system, the, PS (ARM) terminal is divided into two parts: software control and algorithm debugging. The PL (FPGA) side mainly realizes the hardware acceleration of parallel processing of visual algorithm. According to the different resolution output and the configuration mode of the algorithm, the system adopts the design idea of modularization and parametrization. In the process of system design, the camera uses APTINA MT9V034, interface data for high-speed LVDS serial transmission, which can reduce noise and other interference. The polar line geometry correction uses the binary regression polynomial equation to simulate the matrix multiplication, reduces the coordinate storage of two images, and greatly reduces the on-chip hardware logic resources. The pipeline technology is used to calculate the multinomial, which obviously improves the working frequency of the system. Because of the complexity of semi-global stereo matching algorithm and the high requirement for memory, this paper adopts an effective method of computing cost function, and simplifies it to four directions for cost aggregation, designs parallel buffer, and reduces the requirement of memory. Median filtering can filter out mismatching points and stripes caused by dynamic programming. The experimental results show that the system can meet the requirements of real-time video processing. The resolution of the image is 640x480, the maximum disparity search range is 64 pixels, and the frame rate is 60 fps.. For the matching algorithm module, the working frequency of the matching algorithm module can reach 130 MHz, and the real-time video processing of 1280 脳 104 鈮，
本文編號(hào)：2473972
本文鏈接：http://sikaile.net/kejilunwen/ruanjiangongchenglunwen/2473972.html
最近更新
教材專著
