本文選題：大數(shù)據(jù)存儲(chǔ) + 虛擬盤組　； 參考：《湖北工業(yè)大學(xué)》2017年碩士論文

【摘要】：隨著大數(shù)據(jù)儲(chǔ)規(guī)模的日益增大,其高能耗,安全性,數(shù)據(jù)管理已經(jīng)成為一個(gè)無法忽視的問題,因此對(duì)數(shù)據(jù)存儲(chǔ)的節(jié)能,安全,利用率等方面的研究是十分重要的。本文提出了虛擬盤組技術(shù),設(shè)計(jì)出了一種大數(shù)據(jù)存儲(chǔ)系統(tǒng),通過并行處理任務(wù),按照數(shù)據(jù)盤組進(jìn)行數(shù)據(jù)的合并和存儲(chǔ),達(dá)到節(jié)能,安全,管理高效的目的。本文提出大數(shù)據(jù)存儲(chǔ)系統(tǒng)架構(gòu),其中包括硬件設(shè)計(jì)和軟件設(shè)計(jì)。在軟件系統(tǒng)框架中提出了節(jié)點(diǎn)管理,實(shí)現(xiàn)多路數(shù)據(jù)并行存儲(chǔ),大大提高了軟件的存儲(chǔ)效率。在RAID技術(shù)的基礎(chǔ)上,結(jié)合一種Virtual RAID技術(shù),先生成種子文件,后形成數(shù)據(jù)盤組,最后完成數(shù)據(jù)的刻錄存儲(chǔ)。任務(wù)先經(jīng)過預(yù)處理,添加到種子文件中,再調(diào)用刪除算法以確保緩沖區(qū)大小,最后完成數(shù)據(jù)的存儲(chǔ)形成盤組。當(dāng)盤組終結(jié)后,節(jié)點(diǎn)服務(wù)器將數(shù)據(jù)盤組分發(fā)到各個(gè)空閑節(jié)點(diǎn)上,節(jié)點(diǎn)服務(wù)器完成數(shù)據(jù)刻錄存儲(chǔ)。當(dāng)數(shù)據(jù)存儲(chǔ)達(dá)到飽和后轉(zhuǎn)為離線存儲(chǔ),這樣既達(dá)到節(jié)能,安全,同時(shí)也提高了管理效率。為了提高服務(wù)端的兼容性,降低客戶端與服務(wù)端的耦合度,大數(shù)據(jù)存儲(chǔ)系統(tǒng)采用B/S架構(gòu),以數(shù)據(jù)庫(kù)為應(yīng)用層接口,重寫ODBC數(shù)據(jù)庫(kù)連接器,達(dá)到兼容多種數(shù)據(jù)庫(kù),支持多種客戶端界面。本系統(tǒng)經(jīng)過實(shí)際使用驗(yàn)證,在安全性、實(shí)用性、穩(wěn)定性、管理效率等方面得到充分的肯定。大數(shù)據(jù)存儲(chǔ)系統(tǒng)對(duì)于影視、銀行等數(shù)據(jù)存儲(chǔ)有著重要的意義。
[Abstract]:With the increasing size of big data storage, its high energy consumption, security, data management has become a problem that can not be ignored, so the research on energy saving, security and utilization ratio of data storage is very important. In this paper, a virtual disk group technology is proposed, and a big data storage system is designed. By parallel processing task, the data are merged and stored according to the data disk group to achieve the purpose of energy saving, security and efficient management. This paper proposes big data storage system architecture, including hardware design and software design. In the framework of software system, node management is proposed to realize multi-channel data parallel storage, which greatly improves the efficiency of software storage. On the basis of RAID technology, combined with a kind of Virtual RAID technology, a seed file is formed, then a data disk group is formed, and finally the data is recorded and stored. The task is preprocessed and added to the seed file, then the delete algorithm is called to ensure the buffer size, and finally the data is stored to form a disk group. When the disk group ends, the node server distributes the data disk group to each free node, and the node server completes the data burning storage. When the data storage reaches saturation, it is converted to offline storage, which not only achieves energy saving, security, but also improves management efficiency. In order to improve the compatibility of server and reduce the coupling degree between client and server, big data memory system adopts B / S structure, and rewrites ODBC database connector with database as application layer interface, so that it can be compatible with many kinds of databases. Supports multiple client interfaces. The system has been confirmed in safety, practicability, stability and management efficiency. Big data storage system for film and television, banks and other data storage has an important significance.
【學(xué)位授予單位】：湖北工業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 周江;王偉平;孟丹;馬燦;古曉艷;蔣杰;;面向大數(shù)據(jù)分析的分布式文件系統(tǒng)關(guān)鍵技術(shù)[J];計(jì)算機(jī)研究與發(fā)展;2014年02期

2 黃冬梅;杜艷玲;賀琪;;混合云存儲(chǔ)中海洋大數(shù)據(jù)遷移算法的研究[J];計(jì)算機(jī)研究與發(fā)展;2014年01期

3 顧瑜;劉川意;鞠大鵬;汪東升;;基于云存儲(chǔ)的塊級(jí)連續(xù)數(shù)據(jù)保護(hù)系統(tǒng)[J];計(jì)算機(jī)科學(xué)與探索;2014年03期

4 張婧;陳克非;呂林;郭捷;;云存儲(chǔ)中的用戶數(shù)據(jù)安全[J];計(jì)算機(jī)科學(xué)與探索;2013年12期

5 邢晶;熊勁;孫凝暉;馬捷;;一種支持EB級(jí)存儲(chǔ)的可擴(kuò)展存儲(chǔ)空間管理方法[J];計(jì)算機(jī)研究與發(fā)展;2013年08期

6 劉德永;;云計(jì)算和云數(shù)據(jù)管理技術(shù)[J];計(jì)算機(jī)光盤軟件與應(yīng)用;2013年13期

7 邵慧萌;舒紅平;鄭皎凌;許源平;文立玉;;基于分片的高維稀疏數(shù)據(jù)存儲(chǔ)模式優(yōu)化研究[J];計(jì)算機(jī)工程與應(yīng)用;2013年18期

8 陶彩霞;謝曉軍;陳康;郭利榮;劉春;;基于云計(jì)算的移動(dòng)互聯(lián)網(wǎng)大數(shù)據(jù)用戶行為分析引擎設(shè)計(jì)[J];電信科學(xué);2013年03期

9 劉靖宇;鄭軍;李元章;孫志卓;王文明;譚毓安;;混合S-RAID:一種適于連續(xù)數(shù)據(jù)存儲(chǔ)的節(jié)能數(shù)據(jù)布局[J];計(jì)算機(jī)研究與發(fā)展;2013年01期

10 廖彬;于炯;孫華;年梅;;基于存儲(chǔ)結(jié)構(gòu)重配置的分布式存儲(chǔ)系統(tǒng)節(jié)能算法[J];計(jì)算機(jī)研究與發(fā)展;2013年01期

相關(guān)博士學(xué)位論文 前3條

1 白一鳴;基于數(shù)據(jù)挖掘技術(shù)的模糊推理系統(tǒng)設(shè)計(jì)[D];大連海事大學(xué);2013年

2 呂雁飛;閃存數(shù)據(jù)庫(kù)管理系統(tǒng)關(guān)鍵技術(shù)研究[D];北京大學(xué);2013年

3 蔣海波;海量數(shù)據(jù)存儲(chǔ)系統(tǒng)的高可靠性關(guān)鍵技術(shù)研究與應(yīng)用[D];電子科技大學(xué);2013年

，

本文編號(hào)：1825400