本文選題：分布式文件系統(tǒng)　切入點(diǎn)：FUSE　出處：《電子科技大學(xué)》2013年碩士論文　論文類型：學(xué)位論文

【摘要】：自云計(jì)算概念提出以來，作為核心部分之一的云存儲(chǔ)(分布式文件系統(tǒng))也迅速成為研究熱點(diǎn)。與普通的存儲(chǔ)方式不同，云存儲(chǔ)是由大量普通PC形成的存儲(chǔ)集群來提供海量分布式數(shù)據(jù)存儲(chǔ)服務(wù)，通過數(shù)據(jù)冗余的方式來實(shí)現(xiàn)數(shù)據(jù)透明存儲(chǔ)和底層節(jié)點(diǎn)的容錯(cuò)性。如何實(shí)現(xiàn)數(shù)據(jù)的高效存儲(chǔ)是云存儲(chǔ)中的關(guān)鍵技術(shù)之一。 本文通過對(duì)現(xiàn)有分布式文件系統(tǒng)詳細(xì)介紹和全面分析之后，提出了一個(gè)基于私有云的分布式文件系統(tǒng)CStore，CStore系統(tǒng)由大量的元數(shù)據(jù)服務(wù)器和數(shù)據(jù)服務(wù)器節(jié)點(diǎn)等部件組成，向上層用戶提供一個(gè)高效的存儲(chǔ)服務(wù)。最后，結(jié)合項(xiàng)目中的實(shí)際工作，論文設(shè)計(jì)并實(shí)現(xiàn)了CStore系統(tǒng)中基于POSIX語(yǔ)義的Linux客戶端。整個(gè)客戶端所包含的基本工作和創(chuàng)新點(diǎn)如下： 1．FUSE文件系統(tǒng)�？蛻舳藢�(shí)現(xiàn)了一個(gè)基于FUSE協(xié)議的用戶態(tài)文件系統(tǒng)驅(qū)動(dòng)，通過以虛擬磁盤的方式實(shí)現(xiàn)云端數(shù)據(jù)的本地化，原有程序均不修改就能直接運(yùn)行�？蛻舳藘�(nèi)部實(shí)現(xiàn)采用流水線處理模型，每個(gè)功能模塊采用一個(gè)線程進(jìn)行管理，線程之間采用域套接字通信方式和等待隊(duì)列機(jī)制來實(shí)現(xiàn)異步消息的處理。 2．緩存組織。為了減少云端服務(wù)器壓力和通信開銷，客戶端提出了一個(gè)高效的數(shù)據(jù)緩存算法將元數(shù)據(jù)進(jìn)行內(nèi)存緩存和本地化緩存，數(shù)據(jù)進(jìn)行內(nèi)存緩存。在結(jié)構(gòu)組織方面，客戶端采用了樹形目錄的方式管理元數(shù)據(jù)緩存并通過使用紅黑樹和LRU淘汰策略來實(shí)現(xiàn)數(shù)據(jù)分片的隨機(jī)讀寫。在啟動(dòng)時(shí)，客戶端自動(dòng)加載本地元數(shù)據(jù)，在系統(tǒng)退出后，客戶端將所有內(nèi)存元數(shù)據(jù)進(jìn)行本地化存儲(chǔ)。 3．網(wǎng)絡(luò)通信模型�？蛻舳瞬捎肅++實(shí)現(xiàn)了一套基于非阻塞的異步網(wǎng)絡(luò)通信框架來實(shí)現(xiàn)與服務(wù)器通信，并通過線程池來實(shí)現(xiàn)磁盤的異步讀寫功能。與內(nèi)核通信部分，客戶端采用C語(yǔ)言設(shè)計(jì)并實(shí)現(xiàn)該模塊所有基本數(shù)據(jù)結(jié)構(gòu)及業(yè)務(wù)邏輯。 4．提出了基于操作日志的方式來保證數(shù)據(jù)的同步�？蛻舳送ㄟ^定時(shí)觸發(fā)機(jī)制向元數(shù)據(jù)服務(wù)器拉取并本地執(zhí)行操作日志的方式來達(dá)到數(shù)據(jù)的一致。 5．持久化存儲(chǔ)。客戶端提出了一個(gè)基于B+樹的元數(shù)據(jù)持久化存儲(chǔ)模型來存儲(chǔ)內(nèi)存中的元數(shù)據(jù)。通過B+樹建立索引和追加寫策略，，有效解決了元數(shù)據(jù)中目錄存儲(chǔ)和索引節(jié)點(diǎn)存儲(chǔ)問題。 本文將通過文件系統(tǒng)的形式來顯示云端數(shù)據(jù)，該虛擬磁盤中的數(shù)據(jù)來自云端服務(wù)器，也可進(jìn)行擴(kuò)展至P2P客戶端。最后，對(duì)客戶端進(jìn)行壓力測(cè)試與對(duì)比測(cè)試發(fā)現(xiàn)，該客戶端總體性能優(yōu)于同類分布式文件系統(tǒng)。
[Abstract]:Since the concept of cloud computing was put forward, cloud storage (distributed file system), which is one of the core parts of cloud computing, has become a research hotspot. Cloud storage is a storage cluster formed by a large number of ordinary PCs to provide massive distributed data storage services. How to realize the efficient storage of data is one of the key technologies in cloud storage. After the detailed introduction and comprehensive analysis of the existing distributed file systems, this paper proposes a distributed file system based on private cloud, which is composed of a large number of metadata servers and data server nodes. Finally, combined with the actual work in the project, this paper designs and implements the Linux client based on POSIX semantics in CStore system. The basic work and innovation of the whole client are as follows:. 1.FUSE file system. The client implements a user-state file system driver based on FUSE protocol, and localizes cloud data by virtual disk. The original program can be run directly without modification. The client implementation adopts pipeline processing model, each function module is managed by a thread. Threads use domain socket communication and wait queue mechanism to process asynchronous messages. 2. Cache organization. In order to reduce the cloud server pressure and communication overhead, the client proposes an efficient data caching algorithm to cache metadata in memory and local cache, and data to be cached in memory. The client uses the tree directory to manage the metadata cache and realizes the random reading and writing of the data fragments by using the red-black tree and the LRU elimination strategy. At startup, the client automatically loads the local metadata, and after the system exits, the client automatically loads the local metadata. The client stores all memory metadata locally. 3. Network communication model. Client uses C to implement a set of asynchronous network communication framework based on non-blocking to realize communication with server, and to realize asynchronous reading and writing function of disk through thread pool. The client uses C language to design and implement all the basic data structures and business logic of this module. 4. A method based on the operation log is proposed to ensure the synchronization of the data, and the data consistency is achieved by the client pulling the metadata server from the metadata server and executing the operation log locally. 5. persistent storage. The client proposes a metadata persistence storage model based on B-tree to store metadata in memory. The problem of directory storage and index node storage in metadata is solved effectively. This paper will display cloud data in the form of file system. The data in the virtual disk comes from cloud server and can also be extended to P2P client. The overall performance of the client is superior to that of similar distributed file systems.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前5條

1 段翰聰,盧顯良,宋杰;基于EPOLL的單進(jìn)程事件驅(qū)動(dòng)通信服務(wù)器設(shè)計(jì)與分析[J];計(jì)算機(jī)應(yīng)用;2004年10期

2 段翰聰;王勇濤;李林;;EDFUSE:一個(gè)基于異步事件驅(qū)動(dòng)的FUSE用戶級(jí)文件系統(tǒng)框架[J];計(jì)算機(jī)科學(xué);2012年S1期

3 任飛;王念秋;段翰聰;;大規(guī)模分布式存儲(chǔ)系統(tǒng)中數(shù)據(jù)修復(fù)策略的研究[J];互聯(lián)網(wǎng)天地;2013年02期

4 陳康;鄭緯民;;云計(jì)算:系統(tǒng)實(shí)例與研究現(xiàn)狀[J];軟件學(xué)報(bào);2009年05期

5 段翰聰;梅玫;李林;;基于哈希規(guī)則的分布式文件系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];小型微型計(jì)算機(jī)系統(tǒng);2013年06期

本文編號(hào)：1581144