發(fā)布時間：2018-10-07 19:07

【摘要】：隨著互聯(lián)網(wǎng)在生活中的廣泛應用,網(wǎng)絡中的數(shù)據(jù)量呈指數(shù)級增長,IT行業(yè)已經(jīng)從計算時代進入到存儲時代。目前,許多數(shù)據(jù)中心都面臨著系統(tǒng)能耗開銷巨大、存儲資源利用率較低以及存儲成本不斷上升等問題。有數(shù)據(jù)表明,存儲的各種開支已經(jīng)占數(shù)據(jù)中心總成本的30%-70%。而日常產(chǎn)生的數(shù)據(jù)中,經(jīng)常使用的數(shù)據(jù)(熱數(shù)據(jù))只占到整體數(shù)據(jù)量的20%而極少使用或幾乎不用的數(shù)據(jù)(冷數(shù)據(jù))高達80%。在這種情況下,根據(jù)數(shù)據(jù)冷熱特征進行分級存儲,發(fā)揮各種存儲設備優(yōu)點,降低總存儲成本,有十分廣闊的研究和應用前景。對比目前存儲介質(zhì)技術(shù)的發(fā)展,光盤的單位容量價格遠遠低于磁盤,而其讀寫速度又優(yōu)于磁帶。因此,將光盤作為分級存儲系統(tǒng)中的近線或離線存儲設備具有現(xiàn)實意義。本文在分布式文件系統(tǒng)基礎上,引入藍光光盤庫作為近線存儲設備,建立一個“在線——近線”分級存儲的云存儲平臺。在平臺設計和實現(xiàn)的過程中主要工作包括如下內(nèi)容:(1)對當前的數(shù)據(jù)區(qū)分技術(shù)進行研究與分析,在此基礎上,通過對用戶行為日志的分析,提取用戶行為關鍵特征,建立數(shù)據(jù)價值模型。該模型采用指數(shù)平滑法充分考慮各個時刻的數(shù)據(jù)價值,并根據(jù)數(shù)據(jù)價值的高低區(qū)分冷熱數(shù)據(jù)集,該模型能有效地提高云平臺中用戶訪問數(shù)據(jù)時命中在線存儲設備的概率。(2)研究數(shù)據(jù)遷移過程控制策略,實現(xiàn)數(shù)據(jù)在磁盤陣列和藍光存儲兩種不同存儲層次的設備之前自動的、不間斷的遷移。重點分析基于藍光光盤庫的數(shù)據(jù)遷移過程,有效地保證數(shù)據(jù)在遷移過程中的一致性。(3)采用元數(shù)據(jù)服務器與應用服務器分離,前臺頁面與后臺服務分離的分模塊方式,實現(xiàn)平臺各個功能,充分體現(xiàn)低耦合的軟件設計思想。并且進行一系列測試,測試結(jié)果表明平臺實現(xiàn)的數(shù)據(jù)遷移具有有效性和準確性,能夠?qū)崿F(xiàn)數(shù)據(jù)在藍光光盤庫與磁盤之間高效傳輸,保證數(shù)據(jù)的一致性,釋放磁盤空間,并提高數(shù)據(jù)訪問的速度。
[Abstract]:With the wide application of the Internet in daily life, the amount of data in the network has increased exponentially. The IT industry has entered the storage era from the computing era. At present, many data centers are faced with the problems of huge system energy consumption, low utilization of storage resources and rising storage costs. Data show that storage costs account for 30-70% of the total cost of the data center. In the daily data generation, the frequently used data (hot data) accounts for only 20% of the total amount of data, and the data (cold data) that is rarely used or rarely used (cold data) is as high as 80%. In this case, there is a broad research and application prospect for hierarchical storage based on the characteristics of data cooling and heating, to give full play to the advantages of various storage devices and to reduce the total storage cost. Compared with the current development of storage media technology, the unit capacity of CD-ROMs is much lower than that of disks, and the speed of reading and writing is better than that of magnetic tape. Therefore, it is of practical significance to use optical disk as near-line or off-line storage device in hierarchical storage system. Based on the distributed file system, this paper introduces the Blu-ray optical disk library as the near-line storage device, and establishes a "on-line -near-line" hierarchical storage cloud storage platform. In the process of platform design and implementation, the main contents are as follows: (1) Research and analysis of current data differentiation technology, on the basis of which, through the analysis of user behavior log, the key features of user behavior are extracted. Establish data value model. In this model, the exponential smoothing method is used to fully consider the data value at each time, and the cold and hot data sets are distinguished according to the value of the data. This model can effectively improve the probability of users hitting online storage devices when accessing data in cloud platform. (2) Research data migration process control strategy, realize data before disk array and Blu-ray storage two different storage levels of devices automatically, An uninterrupted migration. The data migration process based on Blu-ray optical disk library is analyzed in order to ensure the consistency of data migration effectively. (3) the metadata server is separated from the application server, and the foreground page is separated from the background service. Realize each function of the platform, fully embody the low coupling software design idea. And a series of tests are carried out. The test results show that the data migration implemented by the platform is effective and accurate. It can efficiently transfer the data between the Blu-ray disc library and the disk, ensure the consistency of the data, and release the disk space. And improve the speed of data access.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP333

【參考文獻】

相關期刊論文 前10條

1 張晶;黃小鋒;;一種基于微服務的應用框架[J];計算機系統(tǒng)應用;2016年09期

2 栗輝;唐萌;陳豪;;基于用戶行為分析的網(wǎng)站結(jié)構(gòu)優(yōu)化研究綜述[J];計算機科學;2016年S1期

3 劉建華;張智雄;;保存元數(shù)據(jù)的發(fā)展趨勢研究[J];圖書館雜志;2016年06期

4 朱明哲;劉婷;;藍光技術(shù)應用于廣電網(wǎng)絡儲存系統(tǒng)的可行性[J];中國有線電視;2016年02期

5 杜琳琳;;海量數(shù)據(jù)離線存儲系統(tǒng)研究[J];中國檔案;2016年02期

6 張滬寅;屈乾松;胡瑞蕓;;基于JSON的數(shù)據(jù)交換模型[J];計算機工程與設計;2015年12期

7 陳桂龍;;我國大數(shù)據(jù)發(fā)展狀況[J];中國建設信息化;2015年Z1期

8 羅圣美;張佳程;周揚;程舒晗;;TiStor:基于策略引擎的分級存儲系統(tǒng)[J];小型微型計算機系統(tǒng);2015年08期

9 劉勝久;李天瑞;珠杰;;Zipf定律與網(wǎng)絡信息計量學[J];中文信息學報;2015年04期

10 趙偉東;;電子檔案藍光存儲應用探究[J];檔案學研究;2015年03期

相關博士學位論文 前1條

1 鐘睿明;富云：一種跨越異構(gòu)云平臺的互備可靠云存儲系統(tǒng)的實現(xiàn)機制研究[D];北京郵電大學;2014年

相關碩士學位論文 前7條

1 郭剛;內(nèi)存云分級混合存儲架構(gòu)數(shù)據(jù)遷移策略[D];新疆大學;2016年

2 武琢;在線存儲系統(tǒng)的設計與實現(xiàn)[D];內(nèi)蒙古大學;2015年

3 盧俊華;HDFS存儲機制的分析與研究[D];武漢理工大學;2014年

4 劉曉然;基于文件的數(shù)據(jù)分級存儲的研究與實現(xiàn)[D];昆明理工大學;2013年

5 王艷云;分級存儲系統(tǒng)中數(shù)據(jù)遷移技術(shù)的研究[D];西安建筑科技大學;2012年

6 吳超龍;在線云存儲系統(tǒng)的設計與應用[D];華南理工大學;2010年

7 劉志寬;分級存儲系統(tǒng)中元數(shù)據(jù)管理的設計與實現(xiàn)[D];大連海事大學;2008年

，

本文編號：2255343