【摘要】：隨著科技的發(fā)展,信息技術(shù)已經(jīng)深刻的影響著人類的方方面面,而數(shù)據(jù)海量化給存儲(chǔ)技術(shù)帶來了巨大挑戰(zhàn)。相比傳統(tǒng)的存儲(chǔ)系統(tǒng),分布式存儲(chǔ)系統(tǒng)通過利用大量的小型服務(wù)器甚至個(gè)人PC來組成存儲(chǔ)空間巨大的存儲(chǔ)池,因此具有低成本和高擴(kuò)展的優(yōu)勢。 分布式存儲(chǔ)系統(tǒng)通過存儲(chǔ)冗余數(shù)據(jù)在大量的存儲(chǔ)節(jié)點(diǎn)來提供可靠的數(shù)據(jù)存儲(chǔ)服務(wù),保證文件的可靠性和可用性。副本和糾刪碼是兩種常見的冗余機(jī)制。由于硬件故障、網(wǎng)絡(luò)故障等原因會(huì)造成存儲(chǔ)節(jié)點(diǎn)失效即節(jié)點(diǎn)離開系統(tǒng),這種現(xiàn)象是很常見的。當(dāng)存儲(chǔ)節(jié)點(diǎn)失效時(shí),造成該節(jié)點(diǎn)內(nèi)文件不再被系統(tǒng)所使用。為了維持系統(tǒng)內(nèi)文件的冗余度,需要數(shù)據(jù)修復(fù)機(jī)制來修復(fù)丟失的數(shù)據(jù)。另一方面網(wǎng)絡(luò)中路由器性能越來越強(qiáng)大且能夠?qū)μ囟I(yè)務(wù)提供支持,本文創(chuàng)新的提出使用路由器對P2P存儲(chǔ)提供支持。本文的主要內(nèi)容包括如下兩個(gè)方面： (1)數(shù)據(jù)修復(fù)協(xié)議 針對分布式存儲(chǔ)的數(shù)據(jù)修復(fù)過程,本文創(chuàng)新的提出使用路由器的計(jì)算能力提供支持：對流入的多條數(shù)據(jù)流進(jìn)行編碼,發(fā)送出去一條數(shù)據(jù)流。根據(jù)該基本思路,本文提出了兩種利用路由器的數(shù)據(jù)修復(fù)方案,并設(shè)計(jì)了兩種數(shù)據(jù)修復(fù)協(xié)議：路由器支持的數(shù)據(jù)修復(fù)協(xié)議和樹形路由器支持的數(shù)據(jù)修復(fù)協(xié)議。 在第一種數(shù)據(jù)修復(fù)協(xié)議中,由于路由器編碼使數(shù)據(jù)修復(fù)過程的數(shù)據(jù)流數(shù)量減少,因此能減少整個(gè)過程傳輸?shù)臄?shù)據(jù)量。如果網(wǎng)絡(luò)中有瓶頸鏈路,通過減少瓶頸鏈路的數(shù)據(jù)流,數(shù)據(jù)傳輸速度將變快因而能減少數(shù)據(jù)修復(fù)的時(shí)間。相比于第一種協(xié)議,樹形修復(fù)協(xié)議主要是在修復(fù)之前先構(gòu)造一棵再生樹,選擇帶寬較大的邊來進(jìn)行數(shù)據(jù)傳輸,進(jìn)一步提高數(shù)據(jù)修復(fù)的性能。通過實(shí)驗(yàn)仿真,使用兩種數(shù)據(jù)修復(fù)協(xié)議能極大減少數(shù)據(jù)修復(fù)過程的修復(fù)時(shí)間和修復(fù)數(shù)據(jù)流量。 (2)分布式存儲(chǔ)原型系統(tǒng) 除了利用路由器的計(jì)算能力,路由器的存儲(chǔ)空間也可以被利用：在路由器中緩存存儲(chǔ)系統(tǒng)的數(shù)據(jù),使用戶能夠從最近的路由器上取的所需數(shù)據(jù),減少下載時(shí)間。本文設(shè)計(jì)并實(shí)現(xiàn)了一套基于多核路由器平臺(tái)的P2P存儲(chǔ)系統(tǒng)。系統(tǒng)主要功能包括文件上傳、文件下載、數(shù)據(jù)修復(fù)、節(jié)點(diǎn)檢測等。系統(tǒng)測試表明,在有路由器支持的情況下系統(tǒng)主要性能要比無路由器支持的情況下獲得較大提升。
[Abstract]:With the development of science and technology, information technology has deeply affected all aspects of human beings, and data sea quantization has brought great challenges to storage technology. Compared with traditional storage systems, distributed storage systems make use of a large number of small servers and even personal PC to form storage pools with huge storage space, so they have the advantages of low cost and high expansion. Distributed storage systems provide reliable data storage services by storing redundant data in a large number of storage nodes to ensure the reliability and availability of files. Copy and erasure code are two common redundancy mechanisms. Because of hardware failure and network failure, it is very common to cause storage node failure, that is, node leaving the system. When the storage node fails, the file in the node is no longer used by the system. In order to maintain the redundancy of files in the system, data repair mechanism is needed to repair the missing data. On the other hand, routers in networks are becoming more and more powerful and can support specific services. This paper proposes to use routers to support P2P storage. The main contents of this paper are as follows: (1) the data repair protocol is aimed at the data repair process of distributed storage. This paper proposes to use the computing power of routers to provide support: encoding incoming multiple data streams and sending out one data stream. According to this basic idea, this paper proposes two kinds of data repair schemes using routers, and designs two kinds of data repair protocols: data repair protocols supported by routers and data repair protocols supported by tree routers. In the first data repair protocol, the number of data streams in the data repair process can be reduced because of the router encoding, which can reduce the amount of data transmitted by the whole process. If there is a bottleneck link in the network, by reducing the data flow of the bottleneck link, the speed of data transmission will be faster and the time of data repair can be reduced. Compared with the first protocol, the tree repair protocol is mainly to construct a regenerated tree before the repair, and select the edge with large bandwidth for data transmission to further improve the performance of data repair. Through experimental simulation, using two kinds of data repair protocols can greatly reduce the repair time and repair data flow of the data repair process. (2) in addition to the computing power of the router, the storage space of the router can also be utilized in the distributed storage prototype system: the data of the storage system can be cached in the router, Enables users to retrieve the required data from the nearest router, reducing download time. This paper designs and implements a P2P storage system based on multi-core router platform. The main functions of the system include file upload, file download, data repair, node detection and so on. The system tests show that the main performance of the system with router support is better than that without router support.
【學(xué)位授予單位】：復(fù)旦大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2012
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 田敬;代亞非;;P2P持久存儲(chǔ)研究[J];軟件學(xué)報(bào);2007年06期

，

本文編號(hào)：2322336