本文選題：Hadoop + 安全云存儲(chǔ)��； 參考：《電子科技大學(xué)》2013年碩士論文

【摘要】：隨著信息時(shí)代的數(shù)據(jù)量以PB級(jí)飛速增長，傳統(tǒng)存儲(chǔ)模式在大數(shù)據(jù)存儲(chǔ)方面的局限性逐漸暴露出來，，單一存儲(chǔ)很容易使用戶數(shù)據(jù)丟失，造成巨大損失，越來越多的企業(yè)和研究機(jī)構(gòu)認(rèn)識(shí)到必須有一種新的存儲(chǔ)機(jī)制去改變存儲(chǔ)行業(yè)現(xiàn)狀。云存儲(chǔ)技術(shù)的引入為存儲(chǔ)模式的改變帶來了契機(jī)，國內(nèi)外機(jī)構(gòu)和企業(yè)在云存儲(chǔ)熱潮下紛紛推出自己的云存儲(chǔ)產(chǎn)品和解決方案，但由于對(duì)產(chǎn)品的安全性不夠重視，各類安全事故頻繁發(fā)生，給用戶造成了巨大損失。在此背景下，云存儲(chǔ)系統(tǒng)的安全性問題逐漸成為業(yè)界關(guān)注的熱點(diǎn)。 本文在假設(shè)云計(jì)算提供商可信的前提下，從應(yīng)用安全的角度出發(fā)，通過大量關(guān)鍵技術(shù)的研究，構(gòu)建了一種基于Hadoop的安全云存儲(chǔ)系統(tǒng)。本文首先調(diào)研了國內(nèi)外機(jī)構(gòu)對(duì)云存儲(chǔ)系統(tǒng)和云存儲(chǔ)安全性的研究現(xiàn)狀，在此基礎(chǔ)上對(duì)構(gòu)建云存儲(chǔ)系統(tǒng)涉及到的關(guān)鍵技術(shù)進(jìn)行了深入研究，主要包括HDFS數(shù)據(jù)讀寫原理、虛擬磁盤原理、文件系統(tǒng)過濾驅(qū)動(dòng)原理、HTTPS安全傳輸原理、多線程傳輸原理。隨后結(jié)合實(shí)際情況，對(duì)安全云存儲(chǔ)系統(tǒng)從功能和性能兩個(gè)方面進(jìn)行了需求分析，并提出了系統(tǒng)的總體架構(gòu)，然后根據(jù)總體架構(gòu)從安全客戶端、安全可靠傳輸和服務(wù)器端三個(gè)大的模塊出發(fā)進(jìn)行了詳細(xì)設(shè)計(jì)，主要包括虛擬磁盤和文件系統(tǒng)過濾驅(qū)動(dòng)的兩級(jí)加密機(jī)制設(shè)計(jì)、多線程斷點(diǎn)續(xù)傳設(shè)計(jì)、服務(wù)端的存儲(chǔ)架構(gòu)及數(shù)據(jù)處理模塊設(shè)計(jì)。接下來從代碼實(shí)現(xiàn)的角度對(duì)系統(tǒng)各個(gè)模塊的實(shí)現(xiàn)過程進(jìn)行了詳細(xì)闡述，并對(duì)某些重要功能進(jìn)行了效果展示。最后對(duì)系統(tǒng)從功能上和性能上進(jìn)行了測試，經(jīng)測試構(gòu)建的系統(tǒng)達(dá)到了預(yù)定的功能需求，滿足響應(yīng)性、可靠性、可用性三個(gè)方面的性能指標(biāo)。 本文設(shè)計(jì)和實(shí)現(xiàn)的基于Hadoop的安全云存儲(chǔ)系統(tǒng)在客戶端采用虛擬磁盤和文件系統(tǒng)過濾驅(qū)動(dòng)兩級(jí)加密機(jī)制，通過透明加解密的思想保證了用戶數(shù)據(jù)安全；同步傳輸階段采用HTTPS安全傳輸和多線程斷點(diǎn)續(xù)傳技術(shù)，保證了用戶數(shù)據(jù)傳輸?shù)陌踩院涂煽啃裕环⻊?wù)器端提供監(jiān)控機(jī)制，通過對(duì)存儲(chǔ)集群監(jiān)控、文件同步過程監(jiān)控、服務(wù)器性能監(jiān)控，提高了系統(tǒng)的可維護(hù)性性。
[Abstract]:With the rapid growth of PB level in the information age, the limitations of traditional storage mode in large data storage are gradually exposed. Single storage is easy to use users to lose data and cause huge losses. More and more enterprises and research institutions have realized that there must be a new storage mechanism to change the status of storage industry. The introduction of storage technology has brought an opportunity for the change of storage mode. The domestic and foreign organizations and enterprises have launched their own cloud storage products and solutions in the cloud storage upsurge. However, due to the lack of attention to the security of the products, the frequent occurrence of various security accidents and the huge loss to the users. In this context, the cloud storage system is safe. The problem of full nature has gradually become a hot spot of concern in the industry.
In this paper, a security cloud storage system based on Hadoop is constructed from the perspective of application security and the research of a large number of key technologies. This paper first investigates the status of the research on cloud storage system and cloud storage security at home and abroad, and builds a cloud storage system on the basis of this. The key technologies involved are deeply studied, including the principle of HDFS data reading and writing, the principle of virtual disk, the principle of file system filtering and driving, the principle of HTTPS security transmission and the principle of multithreading transmission. Then, according to the actual situation, the requirements of the security cloud storage system from two aspects of function and ability are analyzed, and the system is put forward. The overall architecture is designed in detail based on the security client, safe and reliable transmission and the three large modules of the server. It mainly includes the design of the two level encryption mechanism of the virtual disk and file system filtering drive, the multi thread breakpoint renewal design, the storage architecture of the service end and the design of the data processing module. From the point of view of code realization, the realization process of each module of the system is elaborated in detail, and some important functions are displayed. Finally, the system is tested in function and performance, and the system has reached the desired functional requirements, full of responsiveness, reliability and availability in three aspects. It can be an indicator.
The security cloud storage system based on Hadoop is designed and implemented in this paper, which uses virtual disk and file system filtering to drive two level encryption mechanism, and ensures the user data security through the transparent encryption and decryption idea. The synchronization transmission phase adopts HTTPS secure transmission and multi thread break point renewal technology to ensure the user data transmission. Security and reliability; the server side provides monitoring mechanism, by monitoring the storage cluster, monitoring the process of file synchronization, monitoring the performance of the server, and improving the maintainability of the system.

【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 楊亞平,李偉琴;基于SSL的數(shù)據(jù)安全傳輸系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];北京航空航天大學(xué)學(xué)報(bào);2001年04期

2 馮常青;師明珠;;多線程文件斷點(diǎn)續(xù)傳[J];電腦編程技巧與維護(hù);2006年08期

3 唐箭;;云存儲(chǔ)系統(tǒng)的分析與應(yīng)用研究[J];電腦知識(shí)與技術(shù);2009年20期

4 陳增強(qiáng),郭嘉琳,劉忠信,袁著祉;具有斷點(diǎn)續(xù)傳功能的文件傳輸系統(tǒng)的設(shè)計(jì)與關(guān)鍵技術(shù)[J];計(jì)算機(jī)工程;2002年12期

5 胡光永;;基于云計(jì)算的數(shù)據(jù)安全存儲(chǔ)策略研究[J];計(jì)算機(jī)測量與控制;2011年10期

6 劉貝;湯斌;;云存儲(chǔ)原理及發(fā)展趨勢[J];科技信息;2011年05期

7 謝峰;劉妹娟;;淺析云存儲(chǔ)信息服務(wù)的發(fā)展現(xiàn)狀及隱憂[J];科技信息;2011年10期

8 傅穎勛;羅圣美;舒繼武;;安全云存儲(chǔ)系統(tǒng)與關(guān)鍵技術(shù)綜述[J];計(jì)算機(jī)研究與發(fā)展;2013年01期

9 李清俊;甘萌;;基于虛擬磁盤的文件加密方法[J];計(jì)算機(jī)工程與設(shè)計(jì);2006年15期

10 王會(huì)波;;安全存儲(chǔ)與云存儲(chǔ)安全[J];信息安全與通信保密;2010年12期

相關(guān)碩士學(xué)位論文 前10條

1 楊坤;基于Hadoop的云存儲(chǔ)系統(tǒng)客戶端的設(shè)計(jì)與實(shí)現(xiàn)[D];華中科技大學(xué);2011年

2 王雁;Windows環(huán)境下的一種安全磁盤[D];四川大學(xué);2006年

3 林海元;基于Windows文件系統(tǒng)微過濾驅(qū)動(dòng)模型的文件透明加解密系統(tǒng)的研究與實(shí)現(xiàn)[D];浙江工業(yè)大學(xué);2009年

4 陳勉;基于新一代過濾驅(qū)動(dòng)框架的安全文件內(nèi)核開發(fā)[D];西南交通大學(xué);2009年

5 劉賢斌;虛擬盤安全存儲(chǔ)研究[D];華中科技大學(xué);2008年

6 羅錦文;一種文件安全存儲(chǔ)系統(tǒng)的的研究與實(shí)現(xiàn)[D];中南大學(xué);2009年

7 陸海龍;數(shù)據(jù)安全存儲(chǔ)系統(tǒng)的研究與實(shí)現(xiàn)[D];復(fù)旦大學(xué);2010年

8 皮長蒂;基于過濾驅(qū)動(dòng)的安全文件系統(tǒng)的研究與實(shí)現(xiàn)[D];電子科技大學(xué);2010年

9 劉文娟;基于Hadoop的文件同步存儲(chǔ)系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];電子科技大學(xué);2012年

10 盧夢僑;網(wǎng)絡(luò)文件遠(yuǎn)程傳輸系統(tǒng)的設(shè)計(jì)與開發(fā)[D];西安石油大學(xué);2011年

本文編號(hào)：1836196