【摘要】：數(shù)據(jù)是當今互聯(lián)網(wǎng)時代最核心最具價值的產(chǎn)物之一,個人在消費和使用互聯(lián)網(wǎng)產(chǎn)品的同時產(chǎn)生了大量需要持久化存儲的數(shù)據(jù),個體存儲設備的解決方式在這樣一個存儲量和存儲要求日益增長的條件下不再能夠滿足個人用戶的要求,因此越來越多的個人用戶成為了云存儲的受眾。云計算幾十年的發(fā)展使得公有云環(huán)境下的存儲技術(shù)已然十分成熟,商業(yè)環(huán)境下誕生了許多優(yōu)秀的云盤產(chǎn)品,但其核心技術(shù)是對外封閉的,而對于這種面向大量個人用戶的公有云盤系統(tǒng),開源社區(qū)也并沒有完善可靠的整體解決方案。本文基于上述原因?qū)⒐性票P涉及的各個關(guān)鍵模塊和技術(shù)點進行逐個研究和設計,提出了一整套可靠的云盤系統(tǒng)實現(xiàn)架構(gòu)。首先對云盤的遠程文件系統(tǒng)操作機制的原理和技術(shù)進行了分析,研究并設計出CFSRPC協(xié)議,給出了協(xié)議的客戶端和服務端的具體格式定義和二者的通信機制,云盤系統(tǒng)通過對該協(xié)議的實現(xiàn),可以透明化遠程文件操作的細節(jié),方便客戶端和服務端上層業(yè)務的實現(xiàn)。與此同時為該協(xié)議設計了事務機制以顆�；募䝼鬏敺⻊�,設計了CSDA算法在協(xié)議級別動態(tài)調(diào)節(jié)文件傳輸?shù)姆謮K大小,增加了該協(xié)議對云盤系統(tǒng)性能的優(yōu)化。接著對云盤系統(tǒng)的核心業(yè)務流程進行分析,針對云盤系統(tǒng)依賴的多個關(guān)鍵模塊進行了分析和整合設計,總結(jié)出以訪問層、管理層、存儲層為核心的整體軟件架構(gòu)�；贑eph分布式文件系統(tǒng)設計核心存儲模塊,研究分析CephFS和MDS的實現(xiàn)原理并根據(jù)云盤系統(tǒng)架構(gòu)和CFSRPC協(xié)議的定義,結(jié)合多路IO復用機制設計實現(xiàn)了Ceph的接入模塊CCD。將云盤系統(tǒng)的請求分化為文件系統(tǒng)操作請求和管理功能操作請求,分別以CCD模塊和管理功能模塊來接受和處理,中間加入LVS+Keepalived的負載機制使得CCD節(jié)點和管理功能節(jié)點支持橫向擴展。參考Keystone的令牌認證機制設計實現(xiàn)了獨立認證系統(tǒng)。以NW.js為框架根據(jù)CFSRPC協(xié)議的定義設計實現(xiàn)了客戶端。最后將整套云盤系統(tǒng)在測試環(huán)境下進行部署,完成了認證流程和云盤基本操作的測試,并對單點CCD進行了性能測試,確認整體系統(tǒng)能夠達到公有云生產(chǎn)環(huán)境的性能要求。
[Abstract]:Data is one of the core and most valuable products in the Internet age. While consuming and using Internet products, individuals have produced a large number of data that need to be persisted and stored at the same time. The solution of individual storage devices can no longer meet the requirements of individual users under the condition of increasing storage capacity and storage requirements, so more and more individual users become the audience of cloud storage. With the development of cloud computing for decades, the storage technology in the public cloud environment has been very mature, and many excellent cloud disk products have been born in the commercial environment, but its core technology is closed to the outside. For this public cloud disk system for a large number of individual users, the open source community does not have a sound and reliable overall solution. Based on the above reasons, the key modules and technical points involved in the public cloud disk are studied and designed one by one, and a set of reliable cloud disk system implementation architecture is proposed. Firstly, the principle and technology of the remote file system operation mechanism of cloud disk are analyzed, and the CFSRPC protocol is studied and designed. The specific format definition of client and server and the communication mechanism of the two protocols are given. Through the implementation of the protocol, the cloud disk system can make the details of the remote file operation transparent and facilitate the implementation of the upper layer business of the client and server. At the same time, the transaction mechanism is designed for the protocol to granulate file transfer service, and the CSDA algorithm is designed to dynamically adjust the block size of file transfer at the protocol level, which increases the performance optimization of the cloud disk system. Then the core business process of cloud disk system is analyzed and integrated design for several key modules of cloud disk system is analyzed and integrated. The overall software architecture with access layer management layer and storage layer as the core is summarized. Based on the design of core storage module of Ceph distributed file system, the realization principle of CephFS and MDS is studied and analyzed. According to the definition of cloud disk system architecture and CFSRPC protocol, the CCD. access module of Ceph is designed and implemented with the multichannel IO multiplexing mechanism. The request of cloud disk system is divided into file system operation request and management function operation request, which are received and processed by CCD module and management function module respectively. The load mechanism of adding LVS Keepalived in the middle enables the CCD node and the management function node to support horizontal expansion. An independent authentication system is designed and implemented with reference to the token authentication mechanism of Keystone. According to the definition of CFSRPC protocol, the client is designed and implemented based on NW.js. Finally, the whole cloud disk system is deployed in the test environment, the authentication process and the basic operation of the cloud disk are tested, and the performance of the single point CCD is tested to confirm that the whole system can meet the performance requirements of the public cloud production environment.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP333

【參考文獻】

相關(guān)期刊論文 前10條

1 穆彥良;徐振明;;Ceph存儲中基于溫度因子的CRUSH算法改進[J];成都信息工程學院學報;2015年06期

2 王文睿;;node-webkit:HTML5桌面應用運行環(huán)境[J];程序員;2014年01期

3 吉慧;;云存儲產(chǎn)品性能測試方法研究[J];電信技術(shù);2013年07期

4 任宇寧;;云計算時代的存儲技術(shù)——云存儲[J];科技傳播;2012年03期

5 李楊;李曙東;黃亮;;Samba服務器的研究與應用[J];中國水運(下半月);2011年06期

6 吳一民;劉偉安;;基于Fuse的用戶態(tài)文件系統(tǒng)的設計[J];微計算機信息;2010年06期

7 王曰芬;章成志;張蓓蓓;吳婷婷;;數(shù)據(jù)清洗研究綜述[J];現(xiàn)代圖書情報技術(shù);2007年12期

8 崔濱;萬旺根;余小清;樓順天;;基于EPOLL機制的LINUX網(wǎng)絡游戲服務器實現(xiàn)方法[J];微計算機信息;2006年21期

9 李春林,王麗芳,蔣澤軍,陳小平;基于身份認證技術(shù)安全體系的研究[J];微電子學與計算機;2005年04期

10 史芳麗,周亞莉;Linux系統(tǒng)中虛擬文件系統(tǒng)內(nèi)核機制研究[J];陜西師范大學學報(自然科學版);2005年01期

相關(guān)會議論文 前1條

1 張畢濤;辛陽;;基于Ceph的海量小文件存儲的優(yōu)化方法[A];第十屆中國通信學會學術(shù)年會論文集[C];2014年

，

本文編號：2396151