【摘要】：隨著互聯(lián)網(wǎng)技術(shù)和信息技術(shù)的不斷發(fā)展，存儲系統(tǒng)中的數(shù)據(jù)量不斷增長并呈海量形式發(fā)展。傳統(tǒng)的依賴本地服務(wù)器的存儲已經(jīng)無法滿足日益增長的存儲需求，因此大規(guī)模的基于分布式存儲的數(shù)據(jù)中心正在迅速增長。分布式存儲系統(tǒng)中，，為了保障數(shù)據(jù)的可用性，容災(zāi)備份是關(guān)鍵問題之一，這需要使用容錯(cuò)策略來實(shí)現(xiàn)系統(tǒng)的高可靠性，特別是存在大量的冷數(shù)據(jù)的分布式存儲系統(tǒng)中，冷數(shù)據(jù)的備份方式需要特別考慮。針對上述問題，在介紹備份容錯(cuò)策略的基礎(chǔ)上，對分布式系統(tǒng)的編碼備份技術(shù)進(jìn)行了分析和研究，并提出了一種基于編碼的分布式備份方案。 通過對現(xiàn)有編碼技術(shù)的研究，提出了一種基于糾刪碼的，適用于含大量冷數(shù)據(jù)分布式系統(tǒng)的一種備份架構(gòu)。這種架構(gòu)在保障數(shù)據(jù)可靠性的同時(shí)，能夠提高系統(tǒng)的存儲利用率，提高系統(tǒng)的負(fù)載均衡性，縮短節(jié)點(diǎn)失效時(shí)的修復(fù)時(shí)間，增加系統(tǒng)靈活性。在系統(tǒng)設(shè)計(jì)上，為了使系統(tǒng)的負(fù)載更為均衡，將存儲節(jié)點(diǎn)中的物理節(jié)點(diǎn)根據(jù)需求分為多個(gè)虛擬節(jié)點(diǎn)并且排序后均勻分布在圓環(huán)結(jié)構(gòu)中，虛擬節(jié)點(diǎn)存儲用戶數(shù)據(jù)。同時(shí)，根據(jù)用戶需求設(shè)計(jì)不同的備份服務(wù)器，選擇存儲組，存儲組由多個(gè)不在同一物理節(jié)點(diǎn)的虛擬節(jié)點(diǎn)構(gòu)成，備份服務(wù)器中的數(shù)據(jù)經(jīng)過分割，糾刪碼編碼后，分散存儲到存儲組內(nèi)的虛擬節(jié)點(diǎn)上，不同的用戶需求可以選擇不同的糾刪碼。處理失效時(shí)，數(shù)據(jù)塊通過解碼的方式恢復(fù)來保證其可靠性，物理節(jié)點(diǎn)通過多存儲組的并行恢復(fù)在保證可靠性的同時(shí)，縮短恢復(fù)時(shí)間。 基于設(shè)計(jì)的模型，對基于糾刪碼的分布式備份方案進(jìn)行了仿真測試，通過節(jié)點(diǎn)選取和數(shù)據(jù)修復(fù)的測試，驗(yàn)證了編碼備份方案能保證數(shù)據(jù)的可靠性，相較于三副本方案，存儲利用率提高36%，較非并行的編碼方案，修復(fù)時(shí)間更短。在測試過程中主要采用了CRS和RS碼兩種糾刪碼進(jìn)行測試，測試表明CRS碼的編解碼效率比RS碼更好，恢復(fù)能達(dá)到高效率。
[Abstract]:With the development of Internet technology and information technology, the amount of data in storage system is increasing and developing in massive form. Traditional storage relying on local servers can no longer meet the increasing demand for storage, so large-scale data centers based on distributed storage are growing rapidly. In distributed storage system, in order to ensure the availability of data, disaster recovery backup is one of the key problems, which needs to use fault-tolerant strategy to achieve high reliability of the system, especially in distributed storage systems with a large amount of cold data. Cold data backup methods need special consideration. Based on the introduction of backup fault-tolerant strategy, the coding backup technology of distributed system is analyzed and studied, and a distributed backup scheme based on coding is proposed. A backup architecture based on erasure codes is proposed for distributed systems with large amount of cold data. This architecture can not only guarantee the reliability of data, but also improve the storage utilization of the system, enhance the load balance of the system, shorten the repair time when the node fails, and increase the flexibility of the system. In the design of the system, in order to balance the load of the system, the physical nodes in the storage nodes are divided into several virtual nodes according to the requirements and distributed uniformly in the ring structure after sorting. The virtual nodes store user data. At the same time, different backup servers are designed according to the needs of users. The storage group is composed of several virtual nodes that are not in the same physical node. The data in the backup server is divided and erasure code is coded. Different user requirements can select different erasure codes. When processing failure, the data block is restored by decoding to ensure its reliability, and the physical node can shorten the recovery time while guaranteeing the reliability by parallel recovery of multiple storage groups. Based on the designed model, the distributed backup scheme based on erasure code is simulated and tested. Through the test of node selection and data repair, it is verified that the coded backup scheme can guarantee the reliability of data, compared with the three-copy scheme. The storage utilization is increased by 36%, and the repair time is shorter than that of the non-parallel coding scheme. In the process of testing, two kinds of erasure codes, CRS and RS codes, are used to test. The results show that the coding and decoding efficiency of CRS codes is better than that of RS codes, and the recovery efficiency is high.
【學(xué)位授予單位】：華中科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP309.3;TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前2條

1 陶鈞;沙基昌;王暉;;基于Erasure Code的分割文件P2P存儲結(jié)構(gòu)設(shè)計(jì)[J];國防科技大學(xué)學(xué)報(bào);2008年06期

2 陶鈞;沙基昌;王暉;;SegEra:一種Erasure Code與分割相結(jié)合的P2P存儲結(jié)構(gòu)[J];小型微型計(jì)算機(jī)系統(tǒng);2009年07期

本文編號：2380766