面向海量存儲系統(tǒng)的分層存儲技術研究
發(fā)布時間：2018-12-21 09:35
【摘要】：隨著數(shù)據(jù)信息量的快速增長，一方面迫切需要有大容量的海量存儲系統(tǒng)來存儲這些海量數(shù)據(jù)信息，但是另一方面，由于在海量存儲系統(tǒng)中往往會出現(xiàn)只有小部分活躍的數(shù)據(jù)文件經(jīng)常被訪問，而有相當大比例的數(shù)據(jù)文件被訪問的次數(shù)比較少的情況，因此必須對海量存儲系統(tǒng)中的數(shù)據(jù)文件進行分層存儲管理。 分層存儲的主要技術思想是：將活躍數(shù)據(jù)文件存放在高性能且價格較昂貴的存儲設備上，而將不活躍的文件存放在容量大但價格低廉的存儲設備中。但是目前的分層存儲技術還存在一些問題，如：典型的文件價值判定方法僅僅考慮數(shù)據(jù)本身的固有屬性以及歷史訪問頻率，很少考慮文件的訪問模式，也沒有考慮將文件存放在存儲系統(tǒng)的哪個層次來加快存儲系統(tǒng)的訪問速度；另外由于文件的活躍度也經(jīng)常是隨著業(yè)務的變化而變化，所以也亟待提出更好的動態(tài)遷移策略來優(yōu)化存儲系統(tǒng)的整體性能。 針對以上問題，本文對海量存儲系統(tǒng)中數(shù)據(jù)文件的分層管理技術進行了較深入的分析和研究，所做的主要研究工作和取得的研究成果如下： 1、分析和研究了海量存儲系統(tǒng)中文件訪問模式和不同存儲設備性能差異，提出了一種自動分層存儲系統(tǒng)（ATSS：Automated Tiered Storage System）的結(jié)構(gòu)。 2、提出了一種基于訪問模式與存儲性能相匹配的文件價值判定方法（FVPSP：FileValuation based on access Pattern and Storage Performance）。該方法的思想是將訪問頻率較高或者隨機讀寫比例較高的數(shù)據(jù)文件放在固態(tài)硬盤SSD上，而將訪問頻率較低或者順序讀寫比例較高的數(shù)據(jù)文件存放在機械硬盤上。 3、針對文件活躍程度不斷變化的存儲系統(tǒng)，，提出了一種基于文件活躍度自適應調(diào)整時間間隔的動態(tài)遷移策略，該策略減少了遷移過程中遷移模塊對正常業(yè)務的影響。 本文對提出的文件價值判定方法和動態(tài)遷移策略進行了性能測試。實驗結(jié)果表明在順序讀寫比例較高的工作負載中，F(xiàn)VPSP文件價值判定方法能夠取得較好的優(yōu)化效果；另外，自適應調(diào)整時間間隔的動態(tài)遷移策略與固定時間間隔的遷移策略相比，能夠在不影響性能的條件下有效地減少文件的遷移次數(shù)。因此本文的研究結(jié)果對海量存儲系統(tǒng)中數(shù)據(jù)文件的分層管理技術研究有一定的參考價值。
[Abstract]:With the rapid growth of data information, on the one hand, there is an urgent need for massive storage system to store these massive data information, but on the other hand, Because in mass storage systems, only a small number of active data files are frequently accessed, while a large proportion of data files are accessed less frequently. Therefore, data files in mass storage system must be managed by hierarchical storage. The main technical idea of hierarchical storage is to store active data files on high performance and expensive storage devices while inactive files are stored in large but inexpensive storage devices. However, there are still some problems in the current hierarchical storage technology, such as: the typical method for determining the value of a file only considers the inherent attributes of the data and the historical access frequency, and rarely considers the access mode of the file. It also does not consider which level of storage system to store files in order to speed up the access speed of storage system; In addition, because the activity of files often changes with the change of business, it is urgent to propose a better dynamic migration strategy to optimize the overall performance of storage system. Aiming at the above problems, this paper makes a deep analysis and research on the hierarchical management technology of data files in mass storage system. The main research work and research results are as follows: 1. This paper analyzes and studies the file access mode and the performance differences of different storage devices in mass storage system, and proposes an automatic hierarchical storage system (ATSS:Automated Tiered Storage System) structure. 2. A file value determination method (FVPSP:FileValuation based on access Pattern and Storage Performance).) based on the matching of access mode and storage performance is proposed. The idea of this method is that the data files with high access frequency or high random reading / writing ratio are placed on the solid state hard disk (SSD), while the data files with low access frequency or high proportion of sequential reading and writing are stored on the mechanical hard disk. 3. For the storage system with changing file activity, a dynamic migration strategy based on adaptive adjusting time interval of file activity is proposed, which reduces the influence of migration module on normal business during migration. This paper tests the performance of the proposed document value determination method and dynamic migration strategy. The experimental results show that the value determination method of FVPSP file can achieve better optimization effect in the workload with high proportion of sequential reading and writing. In addition, the dynamic migration strategy with adaptive time interval can effectively reduce the number of file migrations without affecting the performance compared with that of the fixed time interval migration strategy. Therefore, the results of this paper have a certain reference value for the research of hierarchical management of data files in mass storage system.
【學位授予單位】：華南理工大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333
本文編號：2388732
[Abstract]:With the rapid growth of data information, on the one hand, there is an urgent need for massive storage system to store these massive data information, but on the other hand, Because in mass storage systems, only a small number of active data files are frequently accessed, while a large proportion of data files are accessed less frequently. Therefore, data files in mass storage system must be managed by hierarchical storage. The main technical idea of hierarchical storage is to store active data files on high performance and expensive storage devices while inactive files are stored in large but inexpensive storage devices. However, there are still some problems in the current hierarchical storage technology, such as: the typical method for determining the value of a file only considers the inherent attributes of the data and the historical access frequency, and rarely considers the access mode of the file. It also does not consider which level of storage system to store files in order to speed up the access speed of storage system; In addition, because the activity of files often changes with the change of business, it is urgent to propose a better dynamic migration strategy to optimize the overall performance of storage system. Aiming at the above problems, this paper makes a deep analysis and research on the hierarchical management technology of data files in mass storage system. The main research work and research results are as follows: 1. This paper analyzes and studies the file access mode and the performance differences of different storage devices in mass storage system, and proposes an automatic hierarchical storage system (ATSS:Automated Tiered Storage System) structure. 2. A file value determination method (FVPSP:FileValuation based on access Pattern and Storage Performance).) based on the matching of access mode and storage performance is proposed. The idea of this method is that the data files with high access frequency or high random reading / writing ratio are placed on the solid state hard disk (SSD), while the data files with low access frequency or high proportion of sequential reading and writing are stored on the mechanical hard disk. 3. For the storage system with changing file activity, a dynamic migration strategy based on adaptive adjusting time interval of file activity is proposed, which reduces the influence of migration module on normal business during migration. This paper tests the performance of the proposed document value determination method and dynamic migration strategy. The experimental results show that the value determination method of FVPSP file can achieve better optimization effect in the workload with high proportion of sequential reading and writing. In addition, the dynamic migration strategy with adaptive time interval can effectively reduce the number of file migrations without affecting the performance compared with that of the fixed time interval migration strategy. Therefore, the results of this paper have a certain reference value for the research of hierarchical management of data files in mass storage system.
【學位授予單位】：華南理工大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333
【參考文獻】
相關期刊論文 前1條
1 王印紅;郭相坤;;信息生命周期理論在企業(yè)數(shù)據(jù)中心的應用研究[J];中國管理信息化(綜合版);2007年03期
相關碩士學位論文 前1條
1 宋麗娜;海量存儲系統(tǒng)中數(shù)據(jù)分級存儲的關鍵技術研究[D];國防科學技術大學;2011年
本文編號：2388732
本文鏈接：http://sikaile.net/kejilunwen/jisuanjikexuelunwen/2388732.html
最近更新
教材專著
