【摘要】：隨著社會信息化進程的日益加速,現(xiàn)有通信、金融等領域內(nèi)的業(yè)務數(shù)據(jù)也在不間斷的更新完善,數(shù)據(jù)量呈爆炸式增長,計算機存儲系統(tǒng)的數(shù)據(jù)已達到PB量級,這些數(shù)據(jù)以記錄或者文件的形式存儲在企業(yè)的存儲系統(tǒng)中,隨著時間的推移,數(shù)據(jù)的利用價值也在變化,必然在存儲系統(tǒng)內(nèi)部會產(chǎn)生大量的不常用的數(shù)據(jù),這些數(shù)據(jù)占據(jù)了存儲系統(tǒng)的大量有效存儲空間。 面對日益增長的數(shù)據(jù)量,大多數(shù)企業(yè)想到的是在原有存儲系統(tǒng)的基礎上擴充容量,但是在容量擴充到一定程度后很難繼續(xù)在原有存儲系統(tǒng)上增加容量,等到原有存儲系統(tǒng)的容量不能在容納更多的數(shù)據(jù),企業(yè)不得不花費高成本新增加一套跟現(xiàn)有存儲系統(tǒng)同樣級別的存儲系統(tǒng),從表面上看增加了一套新的存儲系統(tǒng)解決了存儲容量擴充的問題,但是,在大數(shù)據(jù)量的存儲系統(tǒng)磁盤I/0成為數(shù)據(jù)檢索速度的主要瓶頸,這樣做不但不能提高檢索的速度反而在整體上降低了存儲系統(tǒng)的性能；這樣就造成企業(yè)在數(shù)據(jù)處理時效率低、企業(yè)客戶訪問速度慢等一系列的問題,如何解決規(guī)模龐大的記錄形式的數(shù)據(jù)存儲,提高存儲系統(tǒng)中所有存儲資源的利用率,解決傳統(tǒng)存儲在I／0和存儲容量的瓶頸問題,分級存儲有著重要的意義。 該論文首先介紹了分級存儲結構及其必要性和優(yōu)勢所在,分級存儲的國內(nèi)外研究現(xiàn)狀和在實際應用中存在的問題,接著綜述與分級存儲理論息息相關的信息生命周期理論。分析了現(xiàn)有四種基本的數(shù)據(jù)存儲結構及各自的特點,詳細概述了目前典型的三級存儲架構,以及每級存儲架構的特點。 該論文對基本數(shù)據(jù)存儲結構及分級存儲架構分析后,接著分析了先進先出、最優(yōu)置換算法、最近最久未使用、第二次機會等常見的數(shù)據(jù)置換算法基本思想,在分析各算法的各自特點后對比個算法的優(yōu)劣之處,作為現(xiàn)行典型數(shù)據(jù)分級算法的原型,在此基礎上引出對幾種典型數(shù)據(jù)遷移策略的分析,分析了基于存儲空間的高低水位法、基于訪問頻度的CACHE置換算法和基于生命周期的價值評估法,分析對比三種算法的優(yōu)劣。通過分析通信、金融兩種典型行業(yè)應用系統(tǒng)數(shù)據(jù)庫存儲特點,指出兩種行業(yè)中存在的若干問題,接著分析了傳統(tǒng)分級存儲思想下的存儲級別劃分,最后給出銀行業(yè)務系統(tǒng)存儲級別劃分方案及記錄形式數(shù)據(jù)分級存儲系統(tǒng)設計。 該論文提出一種基于綜合頻度指標的數(shù)據(jù)分級存儲方法,該算法綜合考慮了存儲設備的存儲空間、存儲設備性能、數(shù)據(jù)的相關性等因素,試圖通過平衡各個影響因素對數(shù)據(jù)訪問的影響,以達到數(shù)據(jù)最理想的訪問效果,算法在Windows平臺下模擬試驗證明,與兩種典型算法相比,該算法在實現(xiàn)數(shù)據(jù)分級以及平衡存儲系統(tǒng)容量和訪問速度方面更高效。
[Abstract]:With the increasing acceleration of the social information process, the existing business data in the fields of communication, finance and other fields are constantly updated and improved, the amount of data increases explosively, and the data of the computer storage system has reached the order of PB. These data are stored in the enterprise storage system in the form of records or files. As time goes on, the utilization value of the data is also changing, which will inevitably produce a large number of unusual data within the storage system. This data occupies a large amount of efficient storage space in the storage system. Faced with the increasing amount of data, most enterprises are thinking of expanding the capacity on the basis of the original storage system, but it is difficult to continue to increase the capacity on the original storage system after the capacity has been expanded to a certain extent. When the capacity of the original storage system cannot hold more data, the enterprise has to spend high cost to add a new storage system of the same level as the existing storage system. On the surface, a new storage system has been added to solve the problem of storage capacity expansion. However, the disk I / 0, a large storage system with large amount of data, has become the main bottleneck of data retrieval speed. This not only can not improve the speed of retrieval, but also reduces the performance of the storage system as a whole, which results in a series of problems such as low efficiency in data processing, slow access speed of enterprise customers, and so on. It is of great significance to solve the problem of data storage in large scale record form, to improve the utilization of all storage resources in storage system, and to solve the bottleneck problem of traditional storage in I / 0 and storage capacity. This paper first introduces the hierarchical storage structure, its necessity and advantages, the current situation of hierarchical storage research at home and abroad and the problems in practical application, and then summarizes the information life cycle theory, which is closely related to hierarchical storage theory. In this paper, four basic data storage structures and their respective characteristics are analyzed, and the typical three-level storage architecture and the characteristics of each storage architecture are summarized in detail. After analyzing the basic data storage structure and hierarchical storage architecture, this paper analyzes the basic ideas of the data permutation algorithms, such as first-in-first-out, optimal permutation algorithms, the most recently unused data permutation algorithms, the second opportunity, and so on. After analyzing the characteristics of each algorithm, the advantages and disadvantages of each algorithm are compared. As the prototype of the current typical data classification algorithm, the analysis of several typical data migration strategies is introduced, and the high and low water level method based on storage space is analyzed. The CACHE permutation algorithm based on access frequency and the value evaluation method based on life cycle are analyzed and compared. By analyzing the characteristics of database storage in two typical industries such as communication and finance, this paper points out some problems existing in the two industries, and then analyzes the classification of storage levels under the traditional hierarchical storage idea. In the end, the paper gives the classification scheme of bank business system storage level and the design of record form data hierarchical storage system. In this paper, a data hierarchical storage method based on comprehensive frequency index is proposed. The algorithm considers the storage space of storage device, the performance of storage device, the correlation of data and so on. This paper tries to balance the influence of various factors on data access, in order to achieve the optimal data access effect. The simulation results of the algorithm under Windows platform show that the algorithm is compared with two typical algorithms. The algorithm is more efficient in implementing data classification and balancing storage system capacity and access speed.
【學位授予單位】：昆明理工大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關期刊論文 前1條

1 白廣思;;信息資源分級存儲實現(xiàn)信息生命周期管理[J];情報雜志;2007年01期

，

本文編號：2219043