本文選題：大數(shù)據(jù)處理 + 視頻離線處理�。� 參考：《北京郵電大學(xué)》2014年碩士論文

【摘要】：當(dāng)前,智慧城市成為信息時代城市建設(shè)的一個基本目標(biāo),智能視頻安防監(jiān)控是其中重要一環(huán)。視頻監(jiān)控系統(tǒng)已廣泛使用于各行各業(yè),監(jiān)控視頻數(shù)據(jù)已成為一類典型的大數(shù)據(jù),傳統(tǒng)的視頻收集與回放已不能滿足人們對視頻監(jiān)控的需求,我們希望從視頻圖像提取出有效的信息,提供有效的治安防控業(yè)務(wù)信息,因此,如何對監(jiān)控視頻大數(shù)據(jù)進(jìn)行高效的處理成為一個重要研究課題。 本文首先深入分析了Hadoop框架中的三個重要組成部分,即分布式存儲系統(tǒng)HDFS、分布式計(jì)算框架MapReduce和分布式數(shù)據(jù)庫HBase,并總結(jié)了目前一些常用的基于內(nèi)容的視頻處理方法,說明了目前離線視頻處理方法的瓶頸和不足。在分析視頻處理特點(diǎn)的基礎(chǔ)上,提出并實(shí)現(xiàn)了一種基于Hadoop MapReduce計(jì)算框架的分布式離線視頻處理方法,通過設(shè)計(jì)Hadoop視頻處理相關(guān)方法、接口,使Hadoop MapReduce可以像處理文本文件和二進(jìn)制文件那樣處理視頻數(shù)據(jù),解決了Hadoop MapReduce不能直接處理視頻數(shù)據(jù)的問題,這樣,開發(fā)人員在基于Hadoop對視頻數(shù)據(jù)進(jìn)行并行處理時,就可以將更多精力集中在視頻處理的核心算法上。 同時,針對視頻處理時間與視頻復(fù)雜度相關(guān)這一特點(diǎn),本文對Hadoop HDFS的數(shù)據(jù)分布進(jìn)行了重分布設(shè)計(jì)與實(shí)現(xiàn),使Hadoop MapReduce在進(jìn)行此類監(jiān)控視頻大數(shù)據(jù)處理時,系統(tǒng)整體性能有進(jìn)一步優(yōu)化。 在此基礎(chǔ)上,論文對Hadoop MapReduce的數(shù)據(jù)類型進(jìn)行了進(jìn)一步的分析和設(shè)計(jì),并基于此實(shí)現(xiàn)了離線視頻數(shù)據(jù)處理系統(tǒng),集成了分布式視頻轉(zhuǎn)碼、分布式視頻摘要和分布式人員檢索三個應(yīng)用。 測試結(jié)果表明,利用本系統(tǒng)完成海量視頻數(shù)據(jù)處理所需時間開銷大大減少,通過對HDFS的數(shù)據(jù)分布進(jìn)行重分布優(yōu)化,減少了系統(tǒng)I/O,進(jìn)一步提高了Hadoop MapReduce處理視頻應(yīng)用的效率。
[Abstract]:At present, intelligent city has become a basic goal of urban construction in the information age. Intelligent video security monitoring is an important part of it. Video surveillance system has been widely used in all walks of life. Monitoring video data has become a kind of typical large data. Traditional video collection and replay can not meet people's demand for video surveillance. We want to extract effective information from video images and provide effective information on public security and prevention and control. Therefore, how to efficiently handle large data of monitoring video has become an important research topic.
This paper first analyzes three important components in the Hadoop framework, namely, distributed storage system HDFS, distributed computing framework MapReduce and distributed database HBase, and summarizes some commonly used video processing methods based on content. It illustrates the bottleneck and shortcomings of the current off-line frequency processing method. On the basis of rational characteristics, a distributed off-line video processing method based on Hadoop MapReduce computing framework is proposed and implemented. By designing Hadoop video processing related methods and interface, Hadoop MapReduce can handle visual frequency data like text files and binary files, and the Hadoop MapReduce can not be dealt with directly. The problem of video data is so that the developer can focus more on the core algorithm of video processing when it is based on the parallel processing of video data based on Hadoop.
At the same time, in view of the feature of video processing time and video complexity, the redistribution of the data distribution of Hadoop HDFS is designed and implemented, so that the overall performance of the system is further optimized when Hadoop MapReduce is processed for large data processing of this kind of monitoring video.
On this basis, the paper makes a further analysis and design of the data types of Hadoop MapReduce. Based on this, the off-line video data processing system is implemented, which integrates three applications of distributed video transcoding, distributed video summarization and distributed personnel retrieval.
The test results show that the time cost of using this system to complete mass video data processing is greatly reduced. By redistributing the data distribution of HDFS, the system I/O is reduced, and the efficiency of Hadoop MapReduce processing video application is further improved.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2014
【分類號】：TN948.6;TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前4條

1 何明;鄭翔;賴海光;姜峰;;云計(jì)算技術(shù)發(fā)展及應(yīng)用探討[J];電信科學(xué);2010年05期

2 鄭國暉;肖霏;于弼君;;云計(jì)算技術(shù)發(fā)展與應(yīng)用研究[J];硅谷;2011年20期

3 高東海;李文生;張海濤;;基于Hadoop的離線視頻處理技術(shù)研究與實(shí)現(xiàn)[J];軟件;2013年11期

4 青欣;胥光輝;戢瑤;郭霄;;云數(shù)據(jù)庫應(yīng)用研究[J];計(jì)算機(jī)技術(shù)與發(fā)展;2013年05期

，

本文編號：2057500