【摘要】：隨著信息技術(shù)的飛速發(fā)展,特別是移動(dòng)互聯(lián)網(wǎng)、物聯(lián)網(wǎng)等的發(fā)展,數(shù)據(jù)呈現(xiàn)出了爆發(fā)式增長(zhǎng),我們已經(jīng)步入了海量數(shù)據(jù)的時(shí)代。傳統(tǒng)的存儲(chǔ)管理方式已經(jīng)不再滿足當(dāng)前的存儲(chǔ)現(xiàn)狀,如何有效的存儲(chǔ)、管理、維護(hù)這些數(shù)據(jù)已經(jīng)成為了一個(gè)熱點(diǎn)問題。 云存儲(chǔ)技術(shù)的飛速發(fā)展,使得云存儲(chǔ)成為了一種新型的數(shù)據(jù)存儲(chǔ)解決方案。越來越多的開發(fā)者和企業(yè)將數(shù)據(jù)遷移到云端平臺(tái)上,以降低數(shù)據(jù)管理和運(yùn)維成本,并減輕海量數(shù)據(jù)的沖擊,但是云存儲(chǔ)目前還處于發(fā)展階段,各種技術(shù)和相應(yīng)的法律法規(guī)并不成熟和完善,因此存儲(chǔ)在云端的數(shù)據(jù)并不是萬無一失的,很有可能因?yàn)橐恍┩话l(fā)事件導(dǎo)致用戶數(shù)據(jù)的丟失,或者機(jī)密數(shù)據(jù)的信息泄露,考慮到這些因素企業(yè)內(nèi)部比較敏感重要的數(shù)據(jù)是不適合存放在現(xiàn)有的商用云存儲(chǔ)系統(tǒng)之上的。 本文在綜合分析了目前國(guó)內(nèi)外云存儲(chǔ)技術(shù)的發(fā)展現(xiàn)狀,借鑒了目前最穩(wěn)定、最成熟的云存儲(chǔ)產(chǎn)品Amazon S3中的技術(shù)方案,考慮到企業(yè)內(nèi)部現(xiàn)有的硬件存儲(chǔ)設(shè)備,提出了一個(gè)具有高可擴(kuò)展性、高可靠性、兼容不同存儲(chǔ)設(shè)備的分布式云存儲(chǔ)解決方案——基于HDFS的分布式云存儲(chǔ)系統(tǒng)。該系統(tǒng)分為三個(gè)部分：底層數(shù)據(jù)存儲(chǔ)部分、中間邏輯處理部分、前端訪問部分,整個(gè)系統(tǒng)是構(gòu)建在分布式文件系統(tǒng)HDFS之上的,充分利用了其在數(shù)據(jù)災(zāi)備、容錯(cuò)糾錯(cuò)、數(shù)據(jù)恢復(fù)方面的優(yōu)秀表現(xiàn),在底層存儲(chǔ)系統(tǒng)之上設(shè)計(jì)實(shí)現(xiàn)了文件讀寫模塊,在兼容Amazon S3協(xié)議的基礎(chǔ)上,設(shè)計(jì)實(shí)現(xiàn)了面向前端請(qǐng)求的代理模塊、核心業(yè)務(wù)邏輯處理模塊、基于數(shù)據(jù)庫的元數(shù)據(jù)存儲(chǔ)模塊,提供了兩種服務(wù)訪問方式：Web前端瀏覽器訪問、SDK訪問方式,為了確保數(shù)據(jù)請(qǐng)求在傳輸過程中的安全性和完整性,設(shè)計(jì)實(shí)現(xiàn)了安全控制模塊,這樣就構(gòu)建了一個(gè)具有高可擴(kuò)展、高容錯(cuò)、可靠、安全的分布式云存儲(chǔ)系統(tǒng),最后本文完成了整個(gè)云存儲(chǔ)系統(tǒng)的分布式部署和測(cè)試。
[Abstract]:With the rapid development of information technology, especially the development of mobile Internet, Internet of things and so on, the data show explosive growth, we have entered the era of massive data. The traditional storage management method is no longer satisfied with the current storage situation. How to effectively store, manage and maintain these data has become a hot issue. With the rapid development of cloud storage technology, cloud storage has become a new data storage solution. More and more developers and enterprises migrate data to cloud platform to reduce the cost of data management and operation and reduce the impact of massive data, but cloud storage is still in the development stage. Various technologies and corresponding laws and regulations are not mature and perfect, so the data stored in the cloud is not foolproof. It is very likely that some unexpected events will lead to the loss of user data or the disclosure of confidential information. Considering these factors, the more sensitive and important data within the enterprise is not suitable to store on the existing commercial cloud storage system. In this paper, the development status of cloud storage technology at home and abroad is comprehensively analyzed, and the most stable and mature cloud storage product Amazon S3 is used for reference, taking into account the existing hardware storage devices within the enterprise. A distributed cloud storage solution with high scalability, high reliability and compatibility with different storage devices is proposed, which is a distributed cloud storage system based on HDFS. The system is divided into three parts: the underlying data storage part, the middle logic processing part, the front-end access part. The whole system is built on the distributed file system HDFS, which makes full use of the system in data disaster preparedness, fault tolerance and error correction. On the basis of compatible Amazon S3 protocol, the paper designs and implements the proxy module for front-end request and the core business logic processing module. The metadata storage module based on database provides two service access modes: Web front-end browser access SDK access mode. In order to ensure the security and integrity of data request in the process of transmission, the security control module is designed and implemented. In this way, a distributed cloud storage system with high extensibility, high fault tolerance, reliability and security is constructed. Finally, the distributed deployment and test of the whole cloud storage system are completed.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 何海釗;王虎奇;;數(shù)據(jù)庫連接與訪問技術(shù)的研究與應(yīng)用[J];電腦知識(shí)與技術(shù);2009年01期

2 蔡鍵;王樹梅;;基于Google的云計(jì)算實(shí)例分析[J];電腦知識(shí)與技術(shù);2009年25期

3 郭鋒;劉建偉;;基于Socket的局域網(wǎng)絡(luò)通信軟件開發(fā)[J];電子科技;2009年05期

4 李向軍;;基于云計(jì)算的數(shù)據(jù)存儲(chǔ)系統(tǒng)研究[J];硅谷;2010年19期

5 張曉明;姜本臣;;一種AJAX結(jié)合CGI的嵌入式瘦Web服務(wù)器的研究[J];信息技術(shù);2011年12期

6 董勇,周恩強(qiáng),陳娟;基于Infiniband技術(shù)構(gòu)建高性能分布式文件系統(tǒng)-Lustre[J];計(jì)算機(jī)工程與應(yīng)用;2005年22期

7 曾賽峰;朱立谷;李強(qiáng);張福;;企業(yè)級(jí)私有云中的虛擬化實(shí)現(xiàn)[J];計(jì)算機(jī)工程與應(yīng)用;2010年36期

8 祝志敏;修磊;彭新光;;NFS網(wǎng)絡(luò)文件系統(tǒng)技術(shù)在銀行批量收付交易中的應(yīng)用[J];科技情報(bào)開發(fā)與經(jīng)濟(jì);2007年17期

9 吳大勝;;基于超文本標(biāo)記語言的信息隱藏方法研究與實(shí)現(xiàn)[J];軟件導(dǎo)刊;2011年10期

10 馬俊;楊樹軍;;高性能集群文件系統(tǒng)的研究[J];計(jì)算機(jī)工程與設(shè)計(jì);2006年13期

相關(guān)碩士學(xué)位論文 前2條

1 呂寧;基于REST架構(gòu)的Atom Feed存儲(chǔ)集群研究與設(shè)計(jì)[D];北京交通大學(xué);2008年

2 孫楊;基于REST風(fēng)格構(gòu)建Web服務(wù)的研究與應(yīng)用[D];電子科技大學(xué);2009年

，

本文編號(hào)：2193784