海量硬盤數據快速復制技術研究與實現
發(fā)布時間:2018-06-30 17:57
本文選題:快速復制 + 磁盤解析��; 參考:《南京航空航天大學》2012年碩士論文
【摘要】:信息技術日新月異,必然要求信息傳輸速度更快更強,隨著數字化地球的日趨形成,數字化信息存儲量呈現爆炸式發(fā)展,海量磁盤隨之出現,這對海量硬盤數據的快速復制提出了新的要求。本文在深入了解國內外有關數據快速復制技術的基礎上,就目前數據快速復制應用場景與模式進行了深入的分析,針對當前國內外有關硬盤數據復制系統(tǒng)存在的弱點以及復制速率已經達到極限等問題,提出一種基于多塊位多接口并發(fā)輸出的數據復制策略,并就此展開研究。 論文提出基于主機多個數據輸出接口進行并發(fā)數據輸出的數據復制機制,并圍繞該目標展開相關研究并加以實現。主要做的工作如下: (1)構建了磁盤文件系統(tǒng)快速解析模型。通過對硬盤的物理結構特性、數據結構特性、磁盤文件系統(tǒng)、硬盤數據存儲原理進行了深入分析,提出了磁盤文件系統(tǒng)快速識別技術、文件系統(tǒng)驅動技術,實現了磁盤文件系統(tǒng)的快速解析。 (2)提出了“單模接口數據并發(fā)輸出”與“多模接口數據并發(fā)輸出”兩種數據輸出策略,并完成了數據輸出的整體架構建設。設計了數據輸出實行塊位輸出的策略模式,通過測試與實驗得出最優(yōu)塊位劃分策略表,并設計出進行多線數據傳輸、塊位數據流向以及數據輸出整體架構方案。 (3)研究了塊位劃分算法、塊位派發(fā)與接收算法,并對其進行測試,驗證該策略與算法的可行性與有效性。提出數據散列存儲與重組方案策略,并通過自定義數據元結構協(xié)議,對數據元散列存儲進行分析,設計出數據元散列式存儲方案,并對其進行了實現,,然后依據數據元散列存儲協(xié)議,對基于散列式存儲數據元的重組進行研究,設計并實現“快速重組”與“全息重組”兩種散列數據重組模式。 (4)在技術研究基礎上實現了原型系統(tǒng)的開發(fā),并進行了性能測試,達到了預期的目標。在對相關技術與應用策略深入研究的基礎上,根據課題研究目標,對課題提出的海量硬盤數據快速復制系統(tǒng)的應用需求進行分析,進而對系統(tǒng)的硬件設計與軟件集成進行了實現,最后對研制出的系統(tǒng)原型系統(tǒng)進行功能測試與性能分析,測試結果表明了系統(tǒng)的有效性,達到了預期的研究目標。
[Abstract]:With the rapid development of information technology, the speed of information transmission is bound to be faster and stronger. With the formation of digital earth, the amount of digital information storage presents explosive development, and mass disks appear. This puts forward new requirements for the fast copy of massive hard disk data. On the basis of deep understanding of domestic and foreign rapid data replication technology, this paper makes a deep analysis of the current data rapid replication application scenarios and patterns. Aiming at the weakness of hard disk data replication system at home and abroad and the limitation of replication rate, a data replication strategy based on multi-bit and multi-interface concurrent output is proposed and studied. This paper proposes a data replication mechanism for concurrent data output based on multiple host data output interfaces, and carries out related research and implementation around this goal. The main work is as follows: (1) A fast parsing model of disk file system is built. By analyzing the physical structure characteristic, data structure characteristic, disk file system and hard disk data storage principle of hard disk, the paper puts forward the fast recognition technology of disk file system, file system driving technology, The fast parsing of disk file system is realized. (2) two kinds of data output strategies, "single mode interface data concurrent output" and "multimode interface data concurrent output", are proposed, and the whole data output architecture is completed. The strategy mode of block bit output for data output is designed. The optimal block bit partition strategy table is obtained by testing and experiment, and the multi-line data transmission is designed. The whole architecture of block bit data flow and data output. (3) the block bit partition algorithm, block bit dispatch and receiving algorithm are studied and tested to verify the feasibility and effectiveness of the strategy and algorithm. The strategy of data hash storage and recombination is put forward, and the scheme of data element hash storage is designed and implemented by analyzing the data element hash storage through the self-defined data element structure protocol. Then, according to the hash storage protocol of data element, the recombination of data element based on hash storage is studied. Two kinds of hash data recombination modes are designed and implemented. (4) the prototype system is developed on the basis of technical research, and the performance test is carried out, and the expected goal is achieved. On the basis of deep research on related technology and application strategy, according to the research goal, the paper analyzes the application requirements of the mass hard disk data fast copy system put forward by the subject. Then the hardware design and software integration of the system are realized. Finally, the function test and performance analysis of the developed prototype system are carried out. The test results show that the system is effective and achieves the expected research goal.
【學位授予單位】:南京航空航天大學
【學位級別】:碩士
【學位授予年份】:2012
【分類號】:TP333
【參考文獻】
相關期刊論文 前5條
1 余新平,王宏遠;采用DMA技術實現高速數據傳輸的I/O接口卡設計[J];電子技術應用;1999年07期
2 王芳,張江陵,馮丹;RAID的并行I/O調度算法分析[J];計算機工程與科學;2003年02期
3 王玲,錢華林;計算機取證技術及其發(fā)展趨勢[J];軟件學報;2003年09期
4 王瑞峰;;用ADSP2106X外部握手方式DMA實現高速數據采集[J];計算機工程與設計;2008年03期
5 賈惠波,張緒綱;連續(xù)數據記錄中的緩存策略[J];微計算機信息;2001年02期
本文編號:2086359
本文鏈接:http://sikaile.net/kejilunwen/jisuanjikexuelunwen/2086359.html