【摘要】：在生活中,視頻監(jiān)控網(wǎng)、雷達(dá)基站、股票市場(chǎng)等都會(huì)實(shí)時(shí)產(chǎn)生大量獨(dú)立、連續(xù)的數(shù)據(jù)流,這種隨時(shí)間延續(xù)而無(wú)限增長(zhǎng)的動(dòng)態(tài)數(shù)據(jù)集合就是流數(shù)據(jù)。而且隨著經(jīng)濟(jì)的發(fā)展,各行業(yè)的流數(shù)據(jù)的并發(fā)量和流量越來(lái)越大,尤其是視頻安防監(jiān)控領(lǐng)域,傳統(tǒng)的存儲(chǔ)系統(tǒng)由于受到其物理組成以及功能上的局限,已不能滿足海量數(shù)據(jù)存儲(chǔ)的需要。高效能的存儲(chǔ)流數(shù)據(jù)具有重要的現(xiàn)實(shí)意義。為了存儲(chǔ)規(guī)模龐大的流數(shù)據(jù),基于現(xiàn)有的軟硬件技術(shù),適合采用分布式存儲(chǔ)系統(tǒng)。分布式存儲(chǔ)系統(tǒng),是將數(shù)據(jù)分散存儲(chǔ)在多臺(tái)獨(dú)立設(shè)備上的存儲(chǔ)系統(tǒng)。分布式存儲(chǔ)系統(tǒng)具有高可擴(kuò)展、低成本、高性能和易用等優(yōu)勢(shì)。專用于保存流數(shù)據(jù)的分布式流存儲(chǔ)系統(tǒng)不同于其他分布式存儲(chǔ)系統(tǒng),流數(shù)據(jù)時(shí)源源不斷的快速連續(xù)到達(dá),對(duì)網(wǎng)絡(luò)和磁盤形成巨大的負(fù)載壓力,寫任務(wù)多于讀,系統(tǒng)中存儲(chǔ)的舊數(shù)據(jù)幾乎永不變更但是會(huì)在一段時(shí)間后過(guò)期。由于流數(shù)據(jù)的時(shí)間相關(guān)性,流數(shù)據(jù)存儲(chǔ)中通常用時(shí)間戳來(lái)標(biāo)定對(duì)應(yīng)的流數(shù)據(jù),因此具備單調(diào)遞增的時(shí)間序列特征,可用于后期的基于時(shí)間戳或訪問(wèn)的數(shù)據(jù)定位及讀取。此外,流數(shù)據(jù)會(huì)存在存儲(chǔ)生命周期的需求,例如針對(duì)監(jiān)控存儲(chǔ)的原始數(shù)據(jù),會(huì)存在一個(gè)固定的過(guò)期時(shí)間。針對(duì)這種實(shí)際的情況,本文討論了一種用在分布式視頻流存儲(chǔ)系統(tǒng)中的數(shù)據(jù)存儲(chǔ)服務(wù)器上使用的面向時(shí)間序列的流式對(duì)象存儲(chǔ)文件系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn),以滿足高寫入壓力的存儲(chǔ)需求。該文件系統(tǒng),主要包括如下3個(gè)重要的功能點(diǎn):1、實(shí)現(xiàn)了仿文件系統(tǒng)語(yǔ)義供存儲(chǔ)系統(tǒng)開發(fā)者做二次開發(fā);2、高并發(fā)高吞吐讀寫流數(shù)據(jù),同時(shí)保持良好的可靠性;3、通過(guò)多磁盤的優(yōu)化調(diào)度,在滿足系統(tǒng)讀寫任務(wù)的需求前提下,降低系統(tǒng)能耗,延長(zhǎng)磁盤使用壽命,從而達(dá)到節(jié)能環(huán)保。
[Abstract]:In daily life, video surveillance network, radar base station, stock market and so on will produce a large number of independent and continuous data streams in real time. The dynamic data set that grows infinitely with time is stream data. And with the development of economy, the concurrent amount and flow of stream data in various industries are increasing, especially in the field of video security monitoring, the traditional storage system is limited by its physical composition and function. Can no longer meet the need for massive data storage. High performance storage stream data has important practical significance. In order to store large-scale stream data, distributed storage system is suitable based on the existing software and hardware technology. Distributed storage system is a storage system that distributes data on multiple independent devices. Distributed storage systems have the advantages of high scalability, low cost, high performance and easy to use. Unlike other distributed storage systems, distributed stream storage systems that are dedicated to storing streaming data arrive at a constant and rapid rate, creating a huge load on the network and disk, and writing more tasks than reading. The old data stored in the system almost never changes but will expire after a period of time. Due to the temporal correlation of stream data, time stamp is usually used to calibrate the corresponding stream data in stream data storage, so it has monotone increasing feature of time series and can be used to locate and read data based on time stamp or access. In addition, stream data will have a storage lifecycle requirement, such as a fixed expiration time for monitoring stored raw data. In view of this practical situation, this paper discusses the design and implementation of a time series oriented streaming object storage file system, which is used on a data storage server in a distributed video stream storage system. To meet high write pressure storage requirements. The file system mainly includes the following three important function points: 1, the semantic of the file system is realized for the storage system developer to do secondary development, 2, high concurrency, high throughput, read and write stream data, while maintaining good reliability; 3. By optimizing the scheduling of multi-disks, the energy consumption of the system is reduced, and the service life of the disk is prolonged under the premise of satisfying the task of reading and writing of the system, so as to achieve energy saving and environmental protection.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 劉建欽;;安防存儲(chǔ)行業(yè)發(fā)展現(xiàn)狀與趨勢(shì)分析[J];中國(guó)公共安全;2014年14期

2 張慶華;;云存儲(chǔ)技術(shù)在視頻監(jiān)控中的發(fā)展與應(yīng)用[J];中國(guó)安防;2013年08期

3 鄧宏;;網(wǎng)絡(luò)視頻監(jiān)控系統(tǒng)在城市安防中的應(yīng)用[J];中國(guó)公共安全(綜合版);2010年06期

4 鄔建元;顧瑜;鞠大鵬;汪東升;;分布式大規(guī)模監(jiān)控視頻存儲(chǔ)系統(tǒng)THNVR[J];計(jì)算機(jī)工程與應(yīng)用;2009年31期

5 江澤民;;對(duì)中國(guó)能源問(wèn)題的思考[J];上海交通大學(xué)學(xué)報(bào);2008年03期

6 段翰聰,盧顯良,宋杰;基于EPOLL的單進(jìn)程事件驅(qū)動(dòng)通信服務(wù)器設(shè)計(jì)與分析[J];計(jì)算機(jī)應(yīng)用;2004年10期

相關(guān)碩士學(xué)位論文 前7條

1 張宇露;分布式視頻流存儲(chǔ)系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];電子科技大學(xué);2016年

2 郭暉;面向高清視頻監(jiān)控的流存儲(chǔ)系統(tǒng)設(shè)計(jì)與實(shí)現(xiàn)[D];電子科技大學(xué);2015年

3 畢宇;注冊(cè)表文件系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];電子科技大學(xué);2014年

4 詹文翰;大規(guī)模分布式文件系統(tǒng)元數(shù)據(jù)管理子系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];電子科技大學(xué);2013年

5 王磊;一種高性能HDFS存儲(chǔ)平臺(tái)的研究與實(shí)現(xiàn)[D];西安電子科技大學(xué);2013年

6 金奎;基于分布式存儲(chǔ)系統(tǒng)的數(shù)據(jù)安全傳輸?shù)脑O(shè)計(jì)與實(shí)現(xiàn)[D];哈爾濱工業(yè)大學(xué);2009年

7 蘇紅峰;光纖通道交換機(jī)線卡的設(shè)計(jì)與實(shí)現(xiàn)[D];華中科技大學(xué);2006年

，

本文編號(hào)：2328283