本文選題：基于文件的數(shù)據(jù) + 分級存儲管理��； 參考：《昆明理工大學(xué)》2013年碩士論文

【摘要】：數(shù)據(jù)一般分為兩類,一類是可以利用二維表結(jié)構(gòu)表示,使用數(shù)據(jù)庫管理系統(tǒng)進行存儲管理記錄級數(shù)據(jù)；一類則是直接由文件系統(tǒng)管理的基于文件的數(shù)據(jù)。通常在數(shù)據(jù)處理過程中,對基于文件的數(shù)據(jù)的處理遠遠超過對記錄級數(shù)據(jù)的操作,需要存儲的基于文件的數(shù)據(jù)也遠遠多于記錄級數(shù)據(jù)。這對存儲系統(tǒng)的I/O性能和存儲設(shè)備容量提出了更高的要求。同時由于數(shù)據(jù)并非生來就是平等的,伴隨著生命周期的流逝,數(shù)據(jù)實際具有的應(yīng)用價值也不斷降低,存儲價值也不斷變化。由于存儲設(shè)備I/O吞吐率、容量和價格的不同,若不能合理的分配存儲設(shè)備會造成存儲資源的浪費,增加存儲的投入成本。如何解決規(guī)模龐大的基于文件的數(shù)據(jù)存儲,提高存儲系統(tǒng)中所有存儲資源的利用率,解決傳統(tǒng)存儲在I/O和存儲容量的瓶頸問題,分級存儲有著重要的意義。在現(xiàn)有的分級存儲系統(tǒng)中,大部分是針對固定的應(yīng)用場景,而數(shù)據(jù)的重要性是動態(tài)變化的,在不同的應(yīng)用中具有的實際存儲價值也是不同。采用分級存儲意味著數(shù)據(jù)必會在各級存儲設(shè)備間動態(tài)遷移,合理的數(shù)據(jù)遷移需要遵循一定的遷移策略,才能實現(xiàn)分級存儲的價值,對于現(xiàn)有的遷移策略,大都存在著考慮數(shù)據(jù)屬性不全面的情況。針對目前分級存儲存在的問題,本文在詳細分析了分級存儲的研究現(xiàn)狀和存在的問題的基礎(chǔ)了,設(shè)計了基于文件的數(shù)據(jù)分級存儲模型。主要內(nèi)容如下：1.設(shè)計了基于文件的數(shù)據(jù)訪問特性記錄表的數(shù)據(jù)結(jié)構(gòu),用來記錄數(shù)據(jù)的訪問情況。對于需要額外統(tǒng)計的訪問頻度給出了統(tǒng)計方法。系統(tǒng)每次獲得基于文件的數(shù)據(jù)的訪問信息時,不在需要掃描整個存儲系統(tǒng),只需增量掃描記錄表,便可獲得訪問信息。2.提出了遷移對象選擇策略,主要是根據(jù)文件訪問特性記錄表獲得訪問情況,計算數(shù)據(jù)的實際使用價值,確定哪些數(shù)據(jù)是需要被遷移的。3.提出了基于訪問頻度為主要指標(biāo),存儲設(shè)備剩下空間和I/O熱度為輔助指標(biāo)的多指標(biāo)數(shù)據(jù)遷移策略實現(xiàn)數(shù)據(jù)的遷移,以及如何在眾多策略中自適應(yīng)選擇合適的策略。4、在數(shù)據(jù)遷移過程中,需要合理地控制數(shù)據(jù)遷移進程。在有外部中斷請求時,應(yīng)采取相應(yīng)的中斷處理過程,以保障遷移進程的繼續(xù)進行。實驗結(jié)果一方面顯示采用分級存儲能夠有效的解決存儲設(shè)備容量瓶頸問題,使得存儲設(shè)備剩余空間在可控的范圍內(nèi),保證占用空間分布平衡,充分提高了存儲資源的利用率。另一方面顯示采用數(shù)據(jù)遷移控制解決應(yīng)用程序訪問中斷可以提高系統(tǒng)的吞吐率。
[Abstract]:Data can be divided into two categories, one is that it can be represented by two-dimensional table structure, and the database management system is used to store and manage record level data, and the other is file based data which is directly managed by file system. In the process of data processing, the processing of file-based data is far more than the operation of record-level data, and the need to store file-based data is far greater than that of record-level data. This puts forward higher requirements for I / O performance and storage device capacity of storage system. At the same time, because data is not born equal, with the passage of life cycle, the practical application value of data is also decreasing, and the storage value is also changing. Due to the difference of I / O throughput, capacity and price of storage devices, if storage devices can not be allocated properly, it will cause waste of storage resources and increase the cost of storage. How to solve the huge file based data storage, improve the utilization of all storage resources in the storage system, and solve the bottleneck problem of traditional storage in I / O and storage capacity, hierarchical storage has important significance. In the existing hierarchical storage systems, most of them are aimed at fixed application scenarios, and the importance of data is dynamic, and the actual storage value in different applications is also different. The adoption of hierarchical storage means that the data must be dynamically migrated between all levels of storage devices. Reasonable data migration needs to follow certain migration strategies in order to realize the value of hierarchical storage. Most of them have incomplete consideration of data attributes. Aiming at the existing problems of hierarchical storage, this paper analyzes the research status and the existing problems of hierarchical storage in detail, and designs a file based data hierarchical storage model. The main content is as follows: 1. The data structure of the data access characteristic record table based on file is designed to record the data access. A statistical method is given for the access frequency which requires additional statistics. Every time the system acquires the access information of file-based data, it does not need to scan the entire storage system, but only incrementally scans the record table to obtain access information. A migration object selection strategy is proposed, which is mainly based on the file access characteristic record table to obtain access, calculate the practical use value of the data, and determine which data needs to be migrated. 3. This paper proposes a multi-index data migration strategy based on the access frequency as the main index, the storage device remaining space and the I / O heat as the auxiliary index to realize the data migration. And how to adaptively select the appropriate strategy. 4 in the process of data migration, it is necessary to control the process of data migration reasonably. When there is an external interrupt request, the corresponding interrupt processing process should be adopted to ensure the continuation of the migration process. On the one hand, the experimental results show that hierarchical storage can effectively solve the bottleneck problem of storage device capacity, make the remaining space of storage device within a controllable range, ensure a balanced distribution of occupied space, and fully improve the utilization of storage resources. On the other hand, data migration control can improve the throughput of the system.
【學(xué)位授予單位】：昆明理工大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關(guān)期刊論文 前10條

1 羅敏;;數(shù)據(jù)分級存儲策略[J];城建檔案;2009年05期

2 王印紅;郭相坤;;信息生命周期理論在企業(yè)數(shù)據(jù)中心的應(yīng)用研究[J];中國管理信息化(綜合版);2007年03期

3 江菲;湯小春;張曉;趙曉南;;基于價值評估的數(shù)據(jù)遷移策略研究[J];電子設(shè)計工程;2011年07期

4 曹強,謝長生;網(wǎng)絡(luò)存儲系統(tǒng)中I/O請求響應(yīng)時間的研究[J];計算機研究與發(fā)展;2003年08期

5 敖莉;于得水;舒繼武;薛巍;;一種海量數(shù)據(jù)分級存儲系統(tǒng)TH-TS[J];計算機研究與發(fā)展;2011年06期

6 張菁;任怡;廖湘科;;多級存儲數(shù)據(jù)價值評定研究[J];計算機研究與發(fā)展;2012年S1期

7 胡娜;李戰(zhàn)懷;張曉;;分級存儲中自適應(yīng)選擇遷移策略的研究[J];計算機科學(xué);2012年03期

8 呂帥;劉光明;徐凱;劉欣;;海量信息分級存儲數(shù)據(jù)遷移策略研究[J];計算機工程與科學(xué);2009年S1期

9 謝華成;陳向東;;面向云存儲的非結(jié)構(gòu)化數(shù)據(jù)存取[J];計算機應(yīng)用;2012年07期

10 舒繼武;;網(wǎng)絡(luò)存儲專欄 分級存儲與管理[J];中國教育網(wǎng)絡(luò);2007年07期

，

本文編號：1873465