【摘要】：在生活中,視頻監(jiān)控網(wǎng)、雷達(dá)基站、股票市場等都會實時產(chǎn)生大量獨立、連續(xù)的數(shù)據(jù)流,這種隨時間延續(xù)而無限增長的動態(tài)數(shù)據(jù)集合就是流數(shù)據(jù)。而且隨著經(jīng)濟(jì)的發(fā)展,各行業(yè)的流數(shù)據(jù)的并發(fā)量和流量越來越大,尤其是視頻安防監(jiān)控領(lǐng)域,傳統(tǒng)的存儲系統(tǒng)由于受到其物理組成以及功能上的局限,已不能滿足海量數(shù)據(jù)存儲的需要。高效能的存儲流數(shù)據(jù)具有重要的現(xiàn)實意義。為了存儲規(guī)模龐大的流數(shù)據(jù),基于現(xiàn)有的軟硬件技術(shù),適合采用分布式存儲系統(tǒng)。分布式存儲系統(tǒng),是將數(shù)據(jù)分散存儲在多臺獨立設(shè)備上的存儲系統(tǒng)。分布式存儲系統(tǒng)具有高可擴(kuò)展、低成本、高性能和易用等優(yōu)勢。專用于保存流數(shù)據(jù)的分布式流存儲系統(tǒng)不同于其他分布式存儲系統(tǒng),流數(shù)據(jù)時源源不斷的快速連續(xù)到達(dá),對網(wǎng)絡(luò)和磁盤形成巨大的負(fù)載壓力,寫任務(wù)多于讀,系統(tǒng)中存儲的舊數(shù)據(jù)幾乎永不變更但是會在一段時間后過期。由于流數(shù)據(jù)的時間相關(guān)性,流數(shù)據(jù)存儲中通常用時間戳來標(biāo)定對應(yīng)的流數(shù)據(jù),因此具備單調(diào)遞增的時間序列特征,可用于后期的基于時間戳或訪問的數(shù)據(jù)定位及讀取。此外,流數(shù)據(jù)會存在存儲生命周期的需求,例如針對監(jiān)控存儲的原始數(shù)據(jù),會存在一個固定的過期時間。針對這種實際的情況,本文討論了一種用在分布式視頻流存儲系統(tǒng)中的數(shù)據(jù)存儲服務(wù)器上使用的面向時間序列的流式對象存儲文件系統(tǒng)的設(shè)計與實現(xiàn),以滿足高寫入壓力的存儲需求。該文件系統(tǒng),主要包括如下3個重要的功能點:1、實現(xiàn)了仿文件系統(tǒng)語義供存儲系統(tǒng)開發(fā)者做二次開發(fā);2、高并發(fā)高吞吐讀寫流數(shù)據(jù),同時保持良好的可靠性;3、通過多磁盤的優(yōu)化調(diào)度,在滿足系統(tǒng)讀寫任務(wù)的需求前提下,降低系統(tǒng)能耗,延長磁盤使用壽命,從而達(dá)到節(jié)能環(huán)保。
[Abstract]:In daily life, video surveillance network, radar base station, stock market and so on will produce a large number of independent and continuous data streams in real time. The dynamic data set that grows infinitely with time is stream data. And with the development of economy, the concurrent amount and flow of stream data in various industries are increasing, especially in the field of video security monitoring, the traditional storage system is limited by its physical composition and function. Can no longer meet the need for massive data storage. High performance storage stream data has important practical significance. In order to store large-scale stream data, distributed storage system is suitable based on the existing software and hardware technology. Distributed storage system is a storage system that distributes data on multiple independent devices. Distributed storage systems have the advantages of high scalability, low cost, high performance and easy to use. Unlike other distributed storage systems, distributed stream storage systems that are dedicated to storing streaming data arrive at a constant and rapid rate, creating a huge load on the network and disk, and writing more tasks than reading. The old data stored in the system almost never changes but will expire after a period of time. Due to the temporal correlation of stream data, time stamp is usually used to calibrate the corresponding stream data in stream data storage, so it has monotone increasing feature of time series and can be used to locate and read data based on time stamp or access. In addition, stream data will have a storage lifecycle requirement, such as a fixed expiration time for monitoring stored raw data. In view of this practical situation, this paper discusses the design and implementation of a time series oriented streaming object storage file system, which is used on a data storage server in a distributed video stream storage system. To meet high write pressure storage requirements. The file system mainly includes the following three important function points: 1, the semantic of the file system is realized for the storage system developer to do secondary development, 2, high concurrency, high throughput, read and write stream data, while maintaining good reliability; 3. By optimizing the scheduling of multi-disks, the energy consumption of the system is reduced, and the service life of the disk is prolonged under the premise of satisfying the task of reading and writing of the system, so as to achieve energy saving and environmental protection.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 劉建欽;;安防存儲行業(yè)發(fā)展現(xiàn)狀與趨勢分析[J];中國公共安全;2014年14期

2 張慶華;;云存儲技術(shù)在視頻監(jiān)控中的發(fā)展與應(yīng)用[J];中國安防;2013年08期

3 鄧宏;;網(wǎng)絡(luò)視頻監(jiān)控系統(tǒng)在城市安防中的應(yīng)用[J];中國公共安全(綜合版);2010年06期

4 鄔建元;顧瑜;鞠大鵬;汪東升;;分布式大規(guī)模監(jiān)控視頻存儲系統(tǒng)THNVR[J];計算機(jī)工程與應(yīng)用;2009年31期

5 江澤民;;對中國能源問題的思考[J];上海交通大學(xué)學(xué)報;2008年03期

6 段翰聰,盧顯良,宋杰;基于EPOLL的單進(jìn)程事件驅(qū)動通信服務(wù)器設(shè)計與分析[J];計算機(jī)應(yīng)用;2004年10期

相關(guān)碩士學(xué)位論文 前7條

1 張宇露;分布式視頻流存儲系統(tǒng)的設(shè)計與實現(xiàn)[D];電子科技大學(xué);2016年

2 郭暉;面向高清視頻監(jiān)控的流存儲系統(tǒng)設(shè)計與實現(xiàn)[D];電子科技大學(xué);2015年

3 畢宇;注冊表文件系統(tǒng)的設(shè)計與實現(xiàn)[D];電子科技大學(xué);2014年

4 詹文翰;大規(guī)模分布式文件系統(tǒng)元數(shù)據(jù)管理子系統(tǒng)的設(shè)計與實現(xiàn)[D];電子科技大學(xué);2013年

5 王磊;一種高性能HDFS存儲平臺的研究與實現(xiàn)[D];西安電子科技大學(xué);2013年

6 金奎;基于分布式存儲系統(tǒng)的數(shù)據(jù)安全傳輸?shù)脑O(shè)計與實現(xiàn)[D];哈爾濱工業(yè)大學(xué);2009年

7 蘇紅峰;光纖通道交換機(jī)線卡的設(shè)計與實現(xiàn)[D];華中科技大學(xué);2006年

，

本文編號：2328283