【摘要】：近年來(lái)公共安全威脅愈發(fā)嚴(yán)峻,將安防監(jiān)控技術(shù)應(yīng)用到公共安全中刻不容緩�，F(xiàn)有安防監(jiān)控領(lǐng)域?qū)?shù)據(jù)的存儲(chǔ)普遍采用普通分布式存儲(chǔ)系統(tǒng)架構(gòu)。然而,這樣的存儲(chǔ)架構(gòu)下,系統(tǒng)使用多副本策略保證數(shù)據(jù)的可靠性,存儲(chǔ)空間利用率低;此外,多數(shù)系統(tǒng)使用輪詢調(diào)度算法調(diào)度存儲(chǔ)節(jié)點(diǎn),忽略了存儲(chǔ)節(jié)點(diǎn)的負(fù)載均衡性問(wèn)題,導(dǎo)致節(jié)點(diǎn)性能抖動(dòng)較大;而且,在數(shù)據(jù)丟失恢復(fù)任務(wù)中,沒(méi)有考慮視頻流存儲(chǔ)的有效時(shí)長(zhǎng)等因素,數(shù)據(jù)恢復(fù)價(jià)值較低。本系統(tǒng)將分布式存儲(chǔ)架構(gòu)技術(shù)與安防監(jiān)控行業(yè)相結(jié)合,設(shè)計(jì)與實(shí)現(xiàn)了元數(shù)據(jù)服務(wù)器,旨在提高監(jiān)控?cái)?shù)據(jù)的可靠性和系統(tǒng)可擴(kuò)展性,在接入互聯(lián)網(wǎng)的情況下,支持用戶在合法權(quán)限內(nèi),獲取任意攝像頭的任意時(shí)間的數(shù)據(jù)。元數(shù)據(jù)服務(wù)器結(jié)合時(shí)間戳策略,設(shè)計(jì)了基于糾刪碼的分布式存儲(chǔ)系統(tǒng)和卷式元數(shù)據(jù)管理結(jié)構(gòu),實(shí)現(xiàn)了對(duì)存儲(chǔ)任務(wù)和恢復(fù)任務(wù)的均衡調(diào)度。具體研究?jī)?nèi)容如下:1.設(shè)計(jì)和實(shí)現(xiàn)了時(shí)間戳策略來(lái)維護(hù)存儲(chǔ)系統(tǒng)中不同服務(wù)器之間卷信息元數(shù)據(jù)的一致性。存儲(chǔ)系統(tǒng)中的每個(gè)服務(wù)器同時(shí)記錄系統(tǒng)中所有卷數(shù)據(jù)的時(shí)間戳信息,以元數(shù)據(jù)服務(wù)器上的卷時(shí)間戳信息為系統(tǒng)唯一參考標(biāo)準(zhǔn),判斷其余模塊中卷信息元數(shù)據(jù)的有效性。2.設(shè)計(jì)和實(shí)現(xiàn)了以節(jié)點(diǎn)和卷為索引的兩個(gè)維度的元數(shù)據(jù)管理結(jié)構(gòu)。元數(shù)據(jù)服務(wù)器維護(hù)每個(gè)存儲(chǔ)節(jié)點(diǎn)的系統(tǒng)健康狀態(tài)信息,為任務(wù)的均衡調(diào)度提供參考;同時(shí),記錄每個(gè)存儲(chǔ)節(jié)點(diǎn)上數(shù)據(jù)分布情況和卷式數(shù)據(jù)的糾刪碼策略相關(guān)信息,為數(shù)據(jù)丟失后的恢復(fù)任務(wù)提供依據(jù)。同時(shí),支持系統(tǒng)節(jié)點(diǎn)的動(dòng)態(tài)擴(kuò)展。3.設(shè)計(jì)和實(shí)現(xiàn)了基于多目標(biāo)優(yōu)化算法的任務(wù)調(diào)度策略。針對(duì)存儲(chǔ)任務(wù)的分發(fā),目標(biāo)函數(shù)不僅保證節(jié)點(diǎn)的負(fù)載均衡,而且讓不同節(jié)點(diǎn)之間的執(zhí)行任務(wù)數(shù)量差值最小;針對(duì)恢復(fù)任務(wù)優(yōu)先級(jí)的制定,目標(biāo)函數(shù)優(yōu)先選擇數(shù)據(jù)冗余率低、丟失率高、視頻剩余有效時(shí)間長(zhǎng)的卷,提高系統(tǒng)的可靠性。通過(guò)對(duì)分布式視頻流存儲(chǔ)系統(tǒng)進(jìn)行功能和性能測(cè)試,元數(shù)據(jù)服務(wù)器達(dá)到了對(duì)視頻流數(shù)據(jù)存儲(chǔ)、讀取和恢復(fù)等功能的控制管理需求;另外,存儲(chǔ)任務(wù)和恢復(fù)任務(wù)的調(diào)度策略,不僅降低了存儲(chǔ)節(jié)點(diǎn)的負(fù)載失衡機(jī)率,而且延遲了低效恢復(fù)任務(wù)的執(zhí)行,提高了系統(tǒng)恢復(fù)數(shù)據(jù)的后期使用價(jià)值。
[Abstract]:In recent years, the threat of public security is becoming more and more serious, so it is urgent to apply security monitoring technology to public safety. General distributed storage system architecture is widely used in data storage in the field of security monitoring and control. However, in this storage architecture, the system uses multi-replica strategy to ensure the reliability of data, and the utilization of storage space is low. In addition, most systems use polling scheduling algorithm to schedule storage nodes, ignoring the load balancing problem of storage nodes, resulting in greater performance jitter. Moreover, in the task of data loss recovery, the effective time of video stream storage is not taken into account, so the value of data recovery is low. This system combines distributed storage architecture technology with security monitoring industry, and designs and implements metadata server, which aims to improve the reliability of monitoring data and system scalability, in the case of access to the Internet, Allows users to access arbitrary camera data at any time within legitimate permissions. The distributed storage system based on erasure code and the volume metadata management structure are designed in the metadata server combined with the timestamp strategy. The balanced scheduling of storage and recovery tasks is realized. The specific research contents are as follows: 1. A timestamp strategy is designed and implemented to maintain the consistency of volume information metadata between different servers in the storage system. Each server in the storage system simultaneously records the timestamp information of all volume data in the system. The volume timestamp information on the metadata server is regarded as the unique reference standard of the system to judge the validity of the volume information metadata in the other modules. 2. A metadata management structure based on nodes and volumes is designed and implemented. The metadata server maintains the system health status information of each storage node and provides a reference for the balanced scheduling of tasks. At the same time, the data distribution on each storage node and the erasure policy of volume data are recorded to provide the basis for the recovery task after data loss. At the same time, support the dynamic expansion of system nodes. 3. The task scheduling strategy based on multi-objective optimization algorithm is designed and implemented. For the distribution of storage tasks, the objective function not only ensures the load balance of nodes, but also minimizes the difference in the number of execution tasks between different nodes. In order to determine the priority of the recovery task, the objective function first selects the volume with low data redundancy rate, high loss rate and long video residual effective time to improve the reliability of the system. By testing the function and performance of the distributed video stream storage system, the metadata server achieves the control and management requirements for the video stream data storage, reading and recovery functions. In addition, the scheduling strategy of storage task and recovery task not only reduces the load imbalance probability of storage node, but also delays the execution of inefficient recovery task, and improves the later application value of system recovery data.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TN948.6;TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前5條

1 張亞波;蘇藝;;基于Red5服務(wù)器集群負(fù)載均衡調(diào)度算法研究[J];微型機(jī)與應(yīng)用;2013年21期

2 楊_g劍;林波;;分布式存儲(chǔ)系統(tǒng)中一致性哈希算法的研究[J];電腦知識(shí)與技術(shù);2011年22期

3 何佳;;數(shù)據(jù)存儲(chǔ)技術(shù)的比較及發(fā)展趨勢(shì)[J];懷化學(xué)院學(xué)報(bào);2008年11期

4 盧秋波;;視頻監(jiān)控技術(shù)簡(jiǎn)介與發(fā)展趨勢(shì)[J];安防科技;2007年05期

5 韓德志,謝長(zhǎng)生,傅湘林,易法令;一種基于iSCSI的附網(wǎng)存儲(chǔ)服務(wù)器系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];計(jì)算機(jī)研究與發(fā)展;2004年01期

相關(guān)會(huì)議論文 前1條

1 黎燕凌;王磊;;DVR計(jì)算機(jī)數(shù)字監(jiān)控系統(tǒng)[A];中國(guó)工程物理研究院科技年報(bào)（2005）[C];2005年

相關(guān)博士學(xué)位論文 前1條

1 王平;嵌入式計(jì)算機(jī)控制系統(tǒng)容錯(cuò)策略研究[D];中國(guó)科學(xué)院研究生院（上海微系統(tǒng)與信息技術(shù)研究所）;2004年

相關(guān)碩士學(xué)位論文 前5條

1 劉曉東;并行文件系統(tǒng)緩存技術(shù)的研究[D];北京交通大學(xué);2013年

2 黃靖鋒;基于SAN的網(wǎng)絡(luò)存儲(chǔ)技術(shù)研究與應(yīng)用[D];電子科技大學(xué);2010年

3 楊建兵;網(wǎng)絡(luò)存儲(chǔ)安全技術(shù)研究[D];上海交通大學(xué);2007年

4 楊麗娟;網(wǎng)絡(luò)存儲(chǔ)技術(shù)研究及網(wǎng)站存儲(chǔ)系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];西南交通大學(xué);2007年

5 趙恒;星載并行計(jì)算機(jī)硬件系統(tǒng)可靠性設(shè)計(jì)與分析[D];國(guó)防科學(xué)技術(shù)大學(xué);2005年

，

本文編號(hào)：2415832