本文關(guān)鍵詞： 重復(fù)數(shù)據(jù)刪除 存儲備份 JPEG　出處：《成都理工大學》2013年碩士論文　論文類型：學位論文

【摘要】：論文基于作者在“磁盤上實現(xiàn)物理磁帶庫(PTL)功能的虛擬磁帶庫(VTL)”項目開發(fā)中的工作,結(jié)合了作者對存儲領(lǐng)域的一些思考,對移動終端數(shù)字信息結(jié)構(gòu)和重復(fù)數(shù)據(jù)刪除技術(shù)的發(fā)展方向以及帶重復(fù)數(shù)據(jù)刪除技術(shù)的存儲系統(tǒng)的功能設(shè)計、開發(fā)方面等進行了較為系統(tǒng)的探索,實現(xiàn)了一款易部署、廉價的基于linux服務(wù)器的帶重復(fù)數(shù)據(jù)刪除技術(shù)的圖片存儲系統(tǒng)。 重復(fù)數(shù)據(jù)刪除已經(jīng)是數(shù)據(jù)存儲備份領(lǐng)域的必備技術(shù)。隨著信息數(shù)字化的不斷推進,人們需要存儲、傳輸?shù)臄?shù)據(jù)量會越來越多,數(shù)據(jù)信息結(jié)構(gòu)多樣化化也會越來越明顯。3G甚至4G無線網(wǎng)絡(luò)的普及進一步促進了移動終端信息結(jié)構(gòu)的調(diào)整,人們需要從移動終端獲取更豐富的數(shù)據(jù)信息的趨勢也越來越明顯,這種豐富的數(shù)據(jù)信息需求刺激著信息主體由文字轉(zhuǎn)變?yōu)閳D片,甚至進一步轉(zhuǎn)變?yōu)橐曨l。 論文首先介紹了重復(fù)數(shù)據(jù)刪除涉及的關(guān)鍵技術(shù)點及其在實際應(yīng)用中所面臨的問題,然后提出了超大量圖片存儲所面對的不同于一般數(shù)據(jù)存儲的特殊需求,圍繞著重復(fù)數(shù)據(jù)刪除技術(shù)應(yīng)用于圖片存儲的問題進行了深入的探討,其中以建立一個開放的、易實現(xiàn)的、廣泛兼容的圖片存儲系統(tǒng)邏輯結(jié)構(gòu)為重點研究內(nèi)容,以占服務(wù)器市場90%以上的linux操作系統(tǒng)為圖片存儲系統(tǒng)的開發(fā)和部署平臺。利用linux操作系統(tǒng)的虛擬文件系統(tǒng)(VFS)的廣泛支持,以及VFS對網(wǎng)絡(luò)文件系統(tǒng)的支持兼容,開發(fā)實現(xiàn)了一款基于VFS的圖片存儲系統(tǒng)。針對圖片文件的特性,選用CDC算法進行圖片文件的數(shù)據(jù)分塊,并且結(jié)合linux下的磁盤文件系統(tǒng)ext3以及磁盤物理特性對CDC算法的預(yù)設(shè)參數(shù)進行合理的設(shè)置,以期得到適用于圖片存儲的文件分塊算法。針對超大量圖片文件的訪問問題,提出了利用文件名nash值加快文件搜索的方法。 論文以source insight為項目代碼管理工具,以gcc為代碼編譯工具,結(jié)合了圖片文件存儲訪問操作主要以文件而不是數(shù)據(jù)塊為單位的特性,直接利用linux的VFS文件系統(tǒng)的文件操作的概念實現(xiàn)了一款極易部署的圖片重刪存儲系統(tǒng)。該存儲系統(tǒng)是以動態(tài)鏈接庫的形式存在,提供了進行圖片存儲與訪問所需的基本操作接口,并且可以容易的根據(jù)實際應(yīng)用的需要擴展出其他的功能接口。
[Abstract]:Based on the author's work in the project of "Virtual tape library VTL for realizing the function of physical tape library PTL on disk", this paper combines the author's thinking on the field of storage. The development direction of digital information structure and repetitive data deletion technology of mobile terminal and the function design and development of storage system with repetitive data deletion technology are systematically explored. An easy-to-deploy, low-cost image storage system with duplicate data deletion technology based on linux server is implemented. Repeated data deletion is an essential technology in the field of data storage and backup. With the development of information digitization, people need to store and transmit more and more data. The diversification of data information structure will also become more and more obvious. The popularity of 3G and even 4G wireless network will further promote the adjustment of mobile terminal information structure. The trend that people need to obtain more abundant data information from mobile terminal is also more and more obvious. This kind of rich data information demand stimulates the information subject to change from the text to the picture, and even to the video. This paper first introduces the key technologies involved in repeated data deletion and the problems it faces in practical applications, and then puts forward the special requirements of super-mass picture storage which are different from general data storage. In this paper, the application of repeated data deletion in picture storage is deeply discussed, and the emphasis is on the establishment of an open, easy to implement and widely compatible logical structure of picture storage system. The linux operating system, which accounts for more than 90% of the server market, is used as the development and deployment platform of the picture storage system. The virtual file system using the linux operating system is widely supported. As well as the compatibility of VFS to the network file system, a picture storage system based on VFS is developed and implemented. According to the characteristics of the picture file, the CDC algorithm is selected to divide the data of the picture file. And combining the disk file system ext3 under linux and the physical characteristics of the disk, the preset parameters of the CDC algorithm are set reasonably. Aiming at the problem of accessing a large number of picture files, this paper proposes a method to speed up file search by using the nash value of file name. This paper takes source insight as the project code management tool and gcc as the code compiling tool. Combines the image file storage access operation mainly in the file rather than the data block as the unit characteristic. This paper directly utilizes the concept of file operation of VFS file system of linux to realize a picture redelete storage system which is easy to deploy. The storage system is in the form of dynamic link library. Provides the basic operation interface for picture storage and access, and can easily expand other functional interfaces according to the needs of practical applications.
【學位授予單位】：成都理工大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關(guān)期刊論文 前10條

1 段夢博;蔡興旺;;基于內(nèi)容的重復(fù)數(shù)據(jù)刪除技術(shù)的研究[J];電腦知識與技術(shù);2010年22期

2 劉俊輝;;MD5消息摘要算法實現(xiàn)及改進[J];福建電腦;2007年04期

3 范濤;;網(wǎng)絡(luò)存儲技術(shù)的研究與應(yīng)用[J];福建電腦;2008年06期

4 賀毅朝,沈春璞,王立壯,徐紹珍;Rabin密碼系統(tǒng)的分析與實現(xiàn)[J];河北省科學院學報;2002年04期

5 付印金;肖儂;劉芳;;重復(fù)數(shù)據(jù)刪除關(guān)鍵技術(shù)研究進展[J];計算機研究與發(fā)展;2012年01期

6 張洪亮;劉曉潔;李濤;趙奎;張燦;唐偉桐;皮璐琳;;一種支持熱歸檔的異地容災(zāi)系統(tǒng)的設(shè)計與實現(xiàn)[J];計算機工程;2008年08期

7 劉仲;李凱;;一種歸檔存儲系統(tǒng)結(jié)構(gòu)[J];計算機工程;2010年07期

8 徐旭;付威;李芬;吳文陽;張建剛;韓曉明;;基于內(nèi)容地址存儲的文件級備份系統(tǒng)[J];計算機工程;2010年16期

9 王燦;秦志光;馮朝勝;彭靜;;面向重復(fù)數(shù)據(jù)消除的備份數(shù)據(jù)加密方法[J];計算機應(yīng)用;2010年07期

10 廖海生;趙躍龍;;基于MD5算法的重復(fù)數(shù)據(jù)刪除技術(shù)的研究與改進[J];計算機測量與控制;2010年03期

，

本文編號：1450602