【摘要】：當(dāng)今社會(huì),各種數(shù)據(jù)量的急劇膨脹使人們進(jìn)入到大數(shù)據(jù)時(shí)代。大數(shù)據(jù)是指單個(gè)尋址單元可以容納不確定的或任意尺寸的數(shù)據(jù)。規(guī)模非常巨大和復(fù)雜的數(shù)據(jù)集,對(duì)于動(dòng)輒達(dá)到PB、EB或ZB級(jí)別的數(shù)據(jù)量,傳統(tǒng)的數(shù)據(jù)存儲(chǔ)管理機(jī)制處理起來將面臨諸多新的難題。通過對(duì)大數(shù)據(jù)底層存儲(chǔ)研究,尋求一種行之有效的方式來滿足大數(shù)據(jù)高并發(fā)讀寫、海量數(shù)據(jù)的高效率存儲(chǔ)與訪問需求、存儲(chǔ)容量可擴(kuò)展和高可用性的需求,是本研究的主要課題。本研究針對(duì)大數(shù)據(jù)所具有的特性,從數(shù)據(jù)的底層介質(zhì)存儲(chǔ)、文件系統(tǒng)管理及界面訪問形式等層面一一剖析,提出利用FPGA實(shí)現(xiàn)文件系統(tǒng)的方案,通過具體實(shí)例介紹其實(shí)現(xiàn)過程,仿真并調(diào)試。.具體如下：一、底層ATA主機(jī)控制器的設(shè)計(jì)及優(yōu)化。針對(duì)實(shí)驗(yàn)室現(xiàn)有的ATA控制器進(jìn)行重新設(shè)計(jì)與優(yōu)化,完善Ultra DMA讀寫功能,歸納DOS、Windows與Linux操作系統(tǒng)中對(duì)底層硬盤讀寫機(jī)制,總結(jié)出底層硬盤接口的主要參數(shù)并對(duì)接口進(jìn)行重新封裝,使其易與上層文件系統(tǒng)接口。二、FPGA實(shí)現(xiàn)Ext2硬件文件系統(tǒng)。(1)在深入理解FAT 32及Ext 2文件系統(tǒng)的存儲(chǔ)原理的基礎(chǔ)上,抽取文件系統(tǒng)共性,歸納總結(jié)文件基本操作的流程：創(chuàng)建文件、讀文件、寫文件、刪除文件、更新文件等；(2)硬件文件系統(tǒng)與底層ATA主機(jī)控制器直接接口,用VHDL描述Ext 2文件系統(tǒng)的各個(gè)功能模塊及主要操作的狀態(tài)機(jī)描述,代碼通過調(diào)試與仿真；(3)給出兩種方式實(shí)現(xiàn)文件系統(tǒng)的具體方案：狀態(tài)機(jī)與定序器引擎,針對(duì)各自所具備的特點(diǎn)進(jìn)行了詳細(xì)的闡述。三、多模式遠(yuǎn)程訪問的客戶端實(shí)現(xiàn)。(1)B/S模式下通過WEB瀏覽器進(jìn)行遠(yuǎn)程訪問：設(shè)計(jì)前端HTML訪問頁面、編寫CGI程序、在FPGA中的PetaLinux操作系統(tǒng)中添加Boa服務(wù)器、編譯文件系統(tǒng)庫等；(2)C/S模式下通過編寫Socket應(yīng)用程序,實(shí)現(xiàn)文件的遠(yuǎn)程存取過程。本研究在利用FPGA實(shí)現(xiàn)文件系統(tǒng)的具體實(shí)現(xiàn)方案中,提出采用RAM實(shí)現(xiàn)文件系統(tǒng)的基本數(shù)據(jù)結(jié)構(gòu)的方法；將字符查找功能應(yīng)用到硬件電路中,實(shí)現(xiàn)文件名的匹配；硬盤命令控制模塊合理地控制發(fā)往硬盤的命令,提升硬盤的工作效率。用FPGA實(shí)現(xiàn)文件系統(tǒng)可在不通過CPU專門調(diào)度指揮下完成對(duì)數(shù)據(jù)及文件的讀寫操作,實(shí)現(xiàn)用戶數(shù)據(jù)緩存與硬盤的直接交互,節(jié)省資源,縮短文件處理時(shí)間,將文件系統(tǒng)與硬盤控制器作為獨(dú)立的資源節(jié)點(diǎn)可極大的增加系統(tǒng)的可擴(kuò)展性,可為滿足大數(shù)據(jù)需求、實(shí)現(xiàn)資源與服務(wù)的高效調(diào)配提供參考。
[Abstract]:Nowadays, the rapid expansion of all kinds of data makes people enter the era of big data. Big data refers to a single addressable unit that can accommodate data of uncertain or arbitrary size. Because the data set is very large and complex, the traditional data storage management mechanism will face many new problems when it comes to the amount of data which can easily reach the level of PB,EB or ZB. Through the research on big data's underlying storage, we seek an effective way to meet the needs of big data's high concurrent reading and writing, the efficient storage and access of massive data, the scalability of storage capacity and the high availability. Is the main subject of this study. According to the characteristics of big data, this paper analyzes the underlying media storage, file system management and interface access form of data, and puts forward a scheme to realize file system by using FPGA, and introduces its implementation process through concrete examples. Emulate and debug. The details are as follows: first, the design and optimization of the underlying ATA host controller. This paper redesigns and optimizes the existing ATA controller in laboratory, consummates the function of Ultra DMA reading and writing, and induces the reading and writing mechanism of the underlying hard disk in DOS,Windows and Linux operating system. The main parameters of the underlying hard disk interface are summarized and the interface is re-encapsulated to make it easy to interface with the upper file system. Second, FPGA implements Ext2 hardware file system. (1) on the basis of deeply understanding the storage principle of FAT 32 and Ext 2 file system, extract the common character of file system, sum up the flow of file basic operation: create file, read file, write file, Delete files, update files, etc. (2) the hardware file system is directly interfaced with the underlying ATA host controller, each function module of the Ext 2 file system and the state machine description of the main operation are described by VHDL. The code is debugged and simulated; (3) give two ways to realize the file system: state machine and sequencer engine. Third, the client implementation of multi-mode remote access. (1) remote access through WEB browser in B / S mode: design front-end HTML access page, write CGI program, add Boa server to PetaLinux operating system in FPGA, Compile file system library and so on; (2) in C / S mode, the remote access process of files is realized by writing Socket application program. In this paper, in the realization of file system with FPGA, a method of realizing the basic data structure of file system with RAM is put forward, and the function of character searching is applied to the hardware circuit to realize the matching of file name. The hard disk command control module reasonably controls the commands sent to the hard disk and improves the working efficiency of the hard disk. The file system with FPGA can read and write the data and files without the command of CPU, realize the direct interaction between the user's data cache and the hard disk, save the resources and shorten the processing time of the file. Taking the file system and the hard disk controller as independent resource nodes can greatly increase the scalability of the system and provide a reference for meeting the needs of big data and realizing the efficient allocation of resources and services.
【學(xué)位授予單位】：太原理工大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前8條

1 劉輝;劉毅敏;;嵌入式Web服務(wù)器的研究與設(shè)計(jì)[J];數(shù)字通信;2011年04期

2 薛慧敏;武傳華;路后兵;尚東生;;基于MicroBlaze的PetaLinux嵌入式操作系統(tǒng)移植[J];微計(jì)算機(jī)信息;2011年08期

3 朱頌;;linux操作系統(tǒng)中EXT2文件的組成[J];武漢工程大學(xué)學(xué)報(bào);2011年04期

4 鐘柏松;張宇成;周明建;;Linux虛擬文件系統(tǒng)分析[J];計(jì)算機(jī)與現(xiàn)代化;2010年09期

5 張?jiān)~;向勇;鄭志國(guó);;PetaLinux操作系統(tǒng)在MicroBlaze系統(tǒng)中的移植[J];單片機(jī)與嵌入式系統(tǒng)應(yīng)用;2010年07期

6 張斌峰;;IDE接口電子盤實(shí)時(shí)記錄模塊的設(shè)計(jì)和實(shí)現(xiàn)[J];航空計(jì)算技術(shù);2009年02期

7 李曉娟;劉學(xué)斌;胡炳j;程海波;;基于FPGA的Ultra DMA寫控制器的實(shí)現(xiàn)[J];科學(xué)技術(shù)與工程;2007年15期

8 劉丹華,黃道君;利用套接字開發(fā)網(wǎng)絡(luò)通信程序[J];微機(jī)發(fā)展;2003年01期

相關(guān)博士學(xué)位論文 前1條

1 王耀力;基于云架構(gòu)的存儲(chǔ)信息系統(tǒng)研究[D];太原理工大學(xué);2012年

相關(guān)碩士學(xué)位論文 前4條

1 趙振廷;網(wǎng)絡(luò)硬盤存儲(chǔ)系統(tǒng)設(shè)計(jì)與FPGA實(shí)現(xiàn)[D];太原理工大學(xué);2012年

2 楊文一;Windows磁盤信息分析處理系統(tǒng)的研究與實(shí)現(xiàn)[D];廈門大學(xué);2008年

3 李曉娟;基于FPGA的高速數(shù)據(jù)記錄系統(tǒng)的研究[D];中國(guó)科學(xué)院研究生院（西安光學(xué)精密機(jī)械研究所）;2007年

4 秦川;基于Nios處理器的硬盤存儲(chǔ)系統(tǒng)的研究與設(shè)計(jì)[D];南京理工大學(xué);2006年

，

本文編號(hào)：2414365