本文關(guān)鍵詞： 分布式 Hadoop 海量數(shù)據(jù)存儲 云計算　出處：《吉林大學(xué)》2016年碩士論文　論文類型：學(xué)位論文

【摘要】：在互聯(lián)網(wǎng)蓬勃發(fā)展的今天,隨處可以聽到有關(guān)云計算的種種應(yīng)用,過多的耳濡目染似乎也就標志著云時代的真正到來。因此,云端產(chǎn)品自然成為云時代下最為熱門的產(chǎn)物。另外,由于傳統(tǒng)的Web應(yīng)用是將數(shù)據(jù)保存到服務(wù)器下數(shù)據(jù)庫系統(tǒng)中,隨著用戶的數(shù)據(jù)量增多,傳統(tǒng)的web項目面臨著巨大的挑戰(zhàn)。不僅如此,服務(wù)器維護成本也非常高。將數(shù)據(jù)直接保存到云服務(wù)器上面,云存儲不僅可以保障數(shù)據(jù)的安全,為數(shù)據(jù)存儲提供足夠的空間,而且會大大降低維護成本。本文的主要工作是設(shè)計并實現(xiàn)云端存儲。在Hadoop云環(huán)境中,利用Web服務(wù)器程序在web下操作Hadoop分布式文件系統(tǒng)API實現(xiàn)云存儲應(yīng)用。系統(tǒng)采用Struts2,Hibernate3和Spring3三大框架開發(fā)J2EE的MVC三層架構(gòu)應(yīng)用,采用Log4j配置和規(guī)范控制臺輸出的系統(tǒng)日志信息,XFire開發(fā)WebService,提供相應(yīng)的服務(wù),JAVA對Apache下的mail開發(fā)并實現(xiàn)郵件的發(fā)送,頁面處理上主要采用JQuery實現(xiàn)頁面無刷新操作,考慮到該應(yīng)用中用戶信息數(shù)據(jù)量小,因此采用MySQL進行用戶數(shù)據(jù)信息的管理,避免HDFS對文件的循環(huán)遍歷。從系統(tǒng)功能上講,基本實現(xiàn)一個云端數(shù)據(jù)文件存儲系統(tǒng),用戶可以隨時隨地通過瀏覽器訪問并管理自己的數(shù)據(jù)文件,進行上傳,下載,刪除,分享等操作;同時可以管理自己的用戶基本信息。區(qū)別與普通的web系統(tǒng),該系統(tǒng)是分布式系統(tǒng)架構(gòu),訪問速度、響應(yīng)速度明顯快于普通web應(yīng)用,不僅如此,由于Hadoop的HDFS會自動將數(shù)據(jù)文件進行備份,存儲到不同的集群環(huán)境下的從服務(wù)器上,所以不用擔(dān)心一臺服務(wù)器壞了,該機器上的文件就出現(xiàn)無法訪問的情況,因為其它的機器會擔(dān)當起這樣的角色對文件進行管理。
[Abstract]:With the rapid development of the Internet, all kinds of cloud computing applications can be heard everywhere. Too much osmosis seems to mark the real arrival of the cloud age. Cloud products have naturally become the most popular product in the cloud era. In addition, because the traditional Web application is to store data in the database system under the server, with the increase of the user's data, Traditional web projects face enormous challenges. Not only that, server maintenance costs are also very high. To store data directly on the cloud server, cloud storage can not only guarantee the security of data, but also provide enough space for data storage. And will greatly reduce maintenance costs. The main work of this paper is to design and implement cloud storage. In the Hadoop cloud environment, The application of cloud storage is realized by using Web server program to operate Hadoop distributed file system API under web. The system uses Struts 2 + hibernate 3 and Spring3 framework to develop J2EE MVC three-tier architecture application. The system log information output from the Log4j configuration and specification console is used to develop the Web Service. The corresponding service Java is provided to develop the mail under Apache and to send the mail. In the page processing, the JQuery is mainly used to realize the page no refresh operation. Considering the small amount of user information in this application, MySQL is used to manage the user data information to avoid the circular traversal of files by HDFS. In terms of system functions, a cloud data file storage system is basically implemented. Users can access and manage their own data files, upload, download, delete, share and so on at any time and anywhere through the browser; at the same time, they can manage their users' basic information. The system is a distributed system architecture, accessing speed and responding speed is obviously faster than the ordinary web application. Not only that, because the HDFS of Hadoop will automatically backup the data files and store them on the slave server in different cluster environment. So don't worry about the failure of a server, the files on the machine will be inaccessible, because other machines will play such a role in the management of files.
【學(xué)位授予單位】：吉林大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2016
【分類號】：TP333

【參考文獻】

相關(guān)期刊論文 前5條

1 王海榮;劉珂;;基于Hadoop的海量數(shù)據(jù)存儲系統(tǒng)設(shè)計[J];科技通報;2014年09期

2 羅彬;陽靜;袁峗;;數(shù)字圖書館中大數(shù)據(jù)存儲的應(yīng)用研究[J];科技與企業(yè);2013年18期

3 張少敏;李曉強;王保義;;基于Hadoop的智能電網(wǎng)數(shù)據(jù)安全存儲設(shè)計[J];電力系統(tǒng)保護與控制;2013年14期

4 王蘇衛(wèi);;基于Hadoop和Hive的電信行業(yè)數(shù)據(jù)倉庫研究[J];電子技術(shù)與軟件工程;2013年11期

5 張春明;芮建武;何婷婷;;一種Hadoop小文件存儲和讀取的方法[J];計算機應(yīng)用與軟件;2012年11期

，

本文編號：1553165