【摘要】：隨著信息時(shí)代的發(fā)展，海量數(shù)據(jù)的存儲(chǔ)傳輸成為目前必須解決的一個(gè)重要問題。云存儲(chǔ)為海量數(shù)據(jù)的存儲(chǔ)提供了很好的解決方案，但是缺乏標(biāo)準(zhǔn)化的云存儲(chǔ)應(yīng)用程序編程接口，極大的限制了云存儲(chǔ)的應(yīng)用。 缺乏標(biāo)準(zhǔn)化的云存儲(chǔ)應(yīng)用程序編程接口使云網(wǎng)關(guān)成為云存儲(chǔ)必不可少的重要組成部分，云網(wǎng)關(guān)已經(jīng)成為應(yīng)用和云提供商應(yīng)用程序編程接口之間協(xié)議的翻譯，但是由于云網(wǎng)關(guān)很少支持企業(yè)級(jí)的其它服務(wù)，限制了它的應(yīng)用范圍，目前大部分被用于歸檔和備份。云網(wǎng)關(guān)的問題在于它增加了云存儲(chǔ)的復(fù)雜性及限制了云存儲(chǔ)的性能，所以從性能及簡(jiǎn)潔性角度來看，它并不適合作為理想的主要應(yīng)用。 針對(duì)備份歸檔文件中的高冗余問題，云網(wǎng)關(guān)上設(shè)計(jì)實(shí)現(xiàn)重復(fù)數(shù)據(jù)刪除功能，能為云網(wǎng)關(guān)與云平臺(tái)間的通信減少網(wǎng)絡(luò)帶寬同時(shí)減小數(shù)據(jù)在云平臺(tái)上的存儲(chǔ)容量。云網(wǎng)關(guān)上的重復(fù)數(shù)據(jù)刪除功能的設(shè)計(jì)與實(shí)現(xiàn)主要包括在云網(wǎng)關(guān)上對(duì)文件分塊，將分塊文件計(jì)算數(shù)據(jù)指紋，，將數(shù)據(jù)指紋與哈希表中指紋進(jìn)行比對(duì)，丟棄重復(fù)數(shù)據(jù)，將文件信息和非重復(fù)的數(shù)據(jù)分離獨(dú)立保存在Swift云平臺(tái)上。 通過與不進(jìn)行重復(fù)數(shù)據(jù)刪除的云網(wǎng)關(guān)性能對(duì)比表明，在云網(wǎng)關(guān)上進(jìn)行重復(fù)數(shù)據(jù)刪除對(duì)于高冗余數(shù)據(jù)能夠減少76%~91%的重復(fù)數(shù)據(jù)存儲(chǔ)容量，節(jié)省了70%~86%的網(wǎng)絡(luò)帶寬，提高了云網(wǎng)關(guān)的性能，且對(duì)云網(wǎng)關(guān)的響應(yīng)開銷影響較小。
[Abstract]:With the development of the information age, the storage and transmission of massive data has become an important problem that must be solved. Cloud storage provides a good solution for mass data storage, but the lack of standardized cloud storage application programming interface greatly limits the application of cloud storage. The lack of standardized cloud storage application programming interface makes cloud gateway an essential part of cloud storage, and cloud gateway has become the translation of protocol between application and cloud provider application programming interface. However, because cloud gateways rarely support other enterprise-level services, which limits its application, most of them are currently used for archiving and backup. The problem of cloud gateway is that it increases the complexity of cloud storage and limits the performance of cloud storage, so it is not suitable for the ideal main application from the point of view of performance and simplicity. Aiming at the problem of high redundancy in backup archival files, the repeated data deletion function is designed and implemented on cloud gateway, which can reduce the network bandwidth and the storage capacity of data on cloud platform for the communication between cloud gateway and cloud platform. The design and implementation of duplicate data deletion function on cloud gateway mainly include dividing files into blocks, calculating data fingerprints, comparing data fingerprints with fingerprints in hash table, and discarding duplicate data. Separate file information from non-duplicate data and store it independently on the Swift cloud platform. Compared with the cloud gateway without repeated data deletion, it is shown that repeated data deletion on cloud gateway can reduce the storage capacity of 76 / 91% of repetitive data and save 70 / 86% of network bandwidth for high redundant data. The performance of the cloud gateway is improved, and the response overhead of the cloud gateway is less affected.
【學(xué)位授予單位】：華中科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)博士學(xué)位論文 前4條

1 楊天明;網(wǎng)絡(luò)備份中重復(fù)數(shù)據(jù)刪除技術(shù)研究[D];華中科技大學(xué);2010年

2 黃莉;基于語義關(guān)聯(lián)的重復(fù)數(shù)據(jù)清理技術(shù)研究[D];華中科技大學(xué);2011年

3 譚玉娟;數(shù)據(jù)備份系統(tǒng)中數(shù)據(jù)去重技術(shù)研究[D];華中科技大學(xué);2012年

4 王燦;基于在線重復(fù)數(shù)據(jù)消除的海量數(shù)據(jù)處理關(guān)鍵技術(shù)研究[D];電子科技大學(xué);2012年

本文編號(hào)：2186369