本文選題：云存儲 + 云存儲網(wǎng)關(guān)��； 參考：《國防科學(xué)技術(shù)大學(xué)》2012年碩士論文

【摘要】：隨著Internet技術(shù)的飛速發(fā)展，各行各業(yè)產(chǎn)生的數(shù)據(jù)急劇膨脹。傳統(tǒng)的海量信息存儲系統(tǒng)可擴(kuò)展性差，只能通過設(shè)備升級來實(shí)現(xiàn)縱向擴(kuò)展，導(dǎo)致了管理和運(yùn)營成本提高。以分布式文件系統(tǒng)為核心的云存儲系統(tǒng)在存儲容量、可擴(kuò)展性、可靠性方面表現(xiàn)出了特有的優(yōu)勢，在海量數(shù)據(jù)存儲領(lǐng)域的應(yīng)用越來越廣泛。但主流的云存儲系統(tǒng)沒有統(tǒng)一的接口，現(xiàn)有的建立在不同系統(tǒng)之上的應(yīng)用無法直接訪問這些系統(tǒng)，且很難實(shí)現(xiàn)快速遷移。此外，云存儲的數(shù)據(jù)安全問題也是用戶關(guān)注的核心問題。 為了滿足現(xiàn)有應(yīng)用到云存儲平臺的快速遷移和數(shù)據(jù)安全的需求，課題組設(shè)計了云存儲網(wǎng)關(guān)JoinIn。JoinIn將后端的云存儲系統(tǒng)抽象為傳統(tǒng)的文件系統(tǒng)，提供標(biāo)準(zhǔn)的POSIX接口供用戶使用。JoinIn的元數(shù)據(jù)服務(wù)器位于局域網(wǎng)，訪問安全受控，數(shù)據(jù)存儲于后端云存儲系統(tǒng)。 本課題針對由于云存儲架構(gòu)導(dǎo)致的數(shù)據(jù)訪問延遲大、吞吐率低等問題，對云存儲網(wǎng)關(guān)JoinIn的分布式緩存系統(tǒng)進(jìn)行了研究和實(shí)現(xiàn)。JoinIn緩存系統(tǒng)的主要設(shè)計思想是，運(yùn)用緩存“取一次，讀多次”的關(guān)鍵思想，利用訪問的局部性，將用戶訪問頻率高的內(nèi)容保存到離用戶較近的緩存系統(tǒng)中，當(dāng)用戶再次訪問這些數(shù)據(jù)時，就可以從緩存中快速獲取，這樣就避免了和后端云存儲系統(tǒng)的交互，降低了數(shù)據(jù)的傳輸延遲、緩解了后端服務(wù)器的負(fù)載、節(jié)省了帶寬。 本文的主要工作和創(chuàng)新包括： 1)提出了云存儲網(wǎng)關(guān)JoinIn的緩存系統(tǒng)的體系結(jié)構(gòu)，針對內(nèi)存緩存容量有限和易失的特點(diǎn)，提出了內(nèi)存和磁盤組成的兩級緩存結(jié)構(gòu)，增大了緩存容量，實(shí)現(xiàn)了緩存內(nèi)容的持久化存儲。 2)提出了云存儲網(wǎng)關(guān)JoinIn的緩存系統(tǒng)的替換算法——JoinIn_LRU算法，針對經(jīng)典的LRU算法沒有考慮訪問次數(shù)的不足，在LRU基礎(chǔ)上，提出了綜合考慮訪問時間間隔和訪問次數(shù)的算法。 3)設(shè)計和實(shí)現(xiàn)了基于虛擬節(jié)點(diǎn)的一致性哈希緩存集群架構(gòu)：考慮單節(jié)點(diǎn)緩存系統(tǒng)的擴(kuò)展性，在深入研究一致性哈希算法的基礎(chǔ)上，設(shè)計實(shí)現(xiàn)了分布式緩存集群架構(gòu)。 本課題搭建了測試環(huán)境，，對系統(tǒng)進(jìn)行完整的功能測試和性能測試，實(shí)驗(yàn)結(jié)果表明，帶有緩存系統(tǒng)的云存儲系統(tǒng)，讀性能得到了大幅度提高。因此，本文設(shè)計的緩存系統(tǒng)是提高云存儲系統(tǒng)使用體驗(yàn)的有效手段。
[Abstract]:With the rapid development of Internet technology, the data produced by various industries expand rapidly. The traditional mass information storage system has poor scalability and can only achieve vertical expansion through equipment upgrading, which leads to higher management and operation costs. Cloud storage system with distributed file system as the core has shown its unique advantages in storage capacity, scalability and reliability, and has been applied more and more widely in the field of mass data storage. However, the mainstream cloud storage systems do not have a unified interface, existing applications based on different systems can not directly access these systems, and it is difficult to achieve rapid migration. In addition, the data security of cloud storage is also the core concern of users. In order to meet the requirement of fast migration and data security of cloud storage platform, the cloud storage gateway JoinIn.JoinIn abstracts the cloud storage system from the back-end to the traditional file system. Provides standard POSIX interface for users to use .JoinIn metadata server located in the LAN, access security control, data storage in the back-end cloud storage system. Aiming at the problems of large data access delay and low throughput caused by cloud storage architecture, this paper studies the distributed cache system of JoinIn, a cloud storage gateway, and implements the main design idea of .JoinIn cache system. The key idea of "read many times", take advantage of the locality of the access, save the high-frequency content of the user to a cache system that is close to the user. When the user accesses the data again, he can quickly get it from the cache. In this way, the interaction with the back-end cloud storage system is avoided, the data transfer delay is reduced, the load of the back-end server is alleviated, and the bandwidth is saved. The main work and innovations of this paper are as follows: 1) the architecture of the cloud storage gateway JoinIn cache system is proposed. In view of the limited and volatile memory cache capacity, a two-level cache structure composed of memory and disk is proposed. The cache capacity is increased, and the persistent storage of cached content is realized. 2) the replacement algorithm of JoinIn cache system of cloud storage gateway JoinIn LRU algorithm is proposed. The classical LRU algorithm does not consider the shortage of access times. On the basis of LRU, an algorithm considering access interval and number of access is proposed. 3) A cluster architecture of consistent hash cache based on virtual nodes is designed and implemented: the scalability of single node cache system is considered. A distributed cache cluster architecture is designed and implemented on the basis of in-depth research on consistent hash algorithm. In this paper, a testing environment is set up to test the function and performance of the system. The experimental results show that the reading performance of the cloud storage system with buffer system has been greatly improved. Therefore, the cache system designed in this paper is an effective means to improve the experience of cloud storage system.
【學(xué)位授予單位】：國防科學(xué)技術(shù)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2012
【分類號】：TP333


本文編號：2110602