【摘要】：隨著科技的發(fā)展,信息技術已經深刻的影響著人類的方方面面,而數(shù)據(jù)海量化給存儲技術帶來了巨大挑戰(zhàn)。相比傳統(tǒng)的存儲系統(tǒng),分布式存儲系統(tǒng)通過利用大量的小型服務器甚至個人PC來組成存儲空間巨大的存儲池,因此具有低成本和高擴展的優(yōu)勢。 分布式存儲系統(tǒng)通過存儲冗余數(shù)據(jù)在大量的存儲節(jié)點來提供可靠的數(shù)據(jù)存儲服務,保證文件的可靠性和可用性。副本和糾刪碼是兩種常見的冗余機制。由于硬件故障、網絡故障等原因會造成存儲節(jié)點失效即節(jié)點離開系統(tǒng),這種現(xiàn)象是很常見的。當存儲節(jié)點失效時,造成該節(jié)點內文件不再被系統(tǒng)所使用。為了維持系統(tǒng)內文件的冗余度,需要數(shù)據(jù)修復機制來修復丟失的數(shù)據(jù)。另一方面網絡中路由器性能越來越強大且能夠對特定業(yè)務提供支持,本文創(chuàng)新的提出使用路由器對P2P存儲提供支持。本文的主要內容包括如下兩個方面： (1)數(shù)據(jù)修復協(xié)議 針對分布式存儲的數(shù)據(jù)修復過程,本文創(chuàng)新的提出使用路由器的計算能力提供支持：對流入的多條數(shù)據(jù)流進行編碼,發(fā)送出去一條數(shù)據(jù)流。根據(jù)該基本思路,本文提出了兩種利用路由器的數(shù)據(jù)修復方案,并設計了兩種數(shù)據(jù)修復協(xié)議：路由器支持的數(shù)據(jù)修復協(xié)議和樹形路由器支持的數(shù)據(jù)修復協(xié)議。 在第一種數(shù)據(jù)修復協(xié)議中,由于路由器編碼使數(shù)據(jù)修復過程的數(shù)據(jù)流數(shù)量減少,因此能減少整個過程傳輸?shù)臄?shù)據(jù)量。如果網絡中有瓶頸鏈路,通過減少瓶頸鏈路的數(shù)據(jù)流,數(shù)據(jù)傳輸速度將變快因而能減少數(shù)據(jù)修復的時間。相比于第一種協(xié)議,樹形修復協(xié)議主要是在修復之前先構造一棵再生樹,選擇帶寬較大的邊來進行數(shù)據(jù)傳輸,進一步提高數(shù)據(jù)修復的性能。通過實驗仿真,使用兩種數(shù)據(jù)修復協(xié)議能極大減少數(shù)據(jù)修復過程的修復時間和修復數(shù)據(jù)流量。 (2)分布式存儲原型系統(tǒng) 除了利用路由器的計算能力,路由器的存儲空間也可以被利用：在路由器中緩存存儲系統(tǒng)的數(shù)據(jù),使用戶能夠從最近的路由器上取的所需數(shù)據(jù),減少下載時間。本文設計并實現(xiàn)了一套基于多核路由器平臺的P2P存儲系統(tǒng)。系統(tǒng)主要功能包括文件上傳、文件下載、數(shù)據(jù)修復、節(jié)點檢測等。系統(tǒng)測試表明,在有路由器支持的情況下系統(tǒng)主要性能要比無路由器支持的情況下獲得較大提升。
[Abstract]:With the development of science and technology, information technology has deeply affected all aspects of human beings, and data sea quantization has brought great challenges to storage technology. Compared with traditional storage systems, distributed storage systems make use of a large number of small servers and even personal PC to form storage pools with huge storage space, so they have the advantages of low cost and high expansion. Distributed storage systems provide reliable data storage services by storing redundant data in a large number of storage nodes to ensure the reliability and availability of files. Copy and erasure code are two common redundancy mechanisms. Because of hardware failure and network failure, it is very common to cause storage node failure, that is, node leaving the system. When the storage node fails, the file in the node is no longer used by the system. In order to maintain the redundancy of files in the system, data repair mechanism is needed to repair the missing data. On the other hand, routers in networks are becoming more and more powerful and can support specific services. This paper proposes to use routers to support P2P storage. The main contents of this paper are as follows: (1) the data repair protocol is aimed at the data repair process of distributed storage. This paper proposes to use the computing power of routers to provide support: encoding incoming multiple data streams and sending out one data stream. According to this basic idea, this paper proposes two kinds of data repair schemes using routers, and designs two kinds of data repair protocols: data repair protocols supported by routers and data repair protocols supported by tree routers. In the first data repair protocol, the number of data streams in the data repair process can be reduced because of the router encoding, which can reduce the amount of data transmitted by the whole process. If there is a bottleneck link in the network, by reducing the data flow of the bottleneck link, the speed of data transmission will be faster and the time of data repair can be reduced. Compared with the first protocol, the tree repair protocol is mainly to construct a regenerated tree before the repair, and select the edge with large bandwidth for data transmission to further improve the performance of data repair. Through experimental simulation, using two kinds of data repair protocols can greatly reduce the repair time and repair data flow of the data repair process. (2) in addition to the computing power of the router, the storage space of the router can also be utilized in the distributed storage prototype system: the data of the storage system can be cached in the router, Enables users to retrieve the required data from the nearest router, reducing download time. This paper designs and implements a P2P storage system based on multi-core router platform. The main functions of the system include file upload, file download, data repair, node detection and so on. The system tests show that the main performance of the system with router support is better than that without router support.
【學位授予單位】：復旦大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP333

【參考文獻】

相關期刊論文 前1條

1 田敬;代亞非;;P2P持久存儲研究[J];軟件學報;2007年06期

，

本文編號：2322336