本文選題：云存儲　切入點(diǎn)：個人云存儲　出處：《浙江大學(xué)》2015年碩士論文

【摘要】：云存儲服務(wù)作為云計算的重要內(nèi)容,是解決海量用戶訪問海量數(shù)據(jù)的關(guān)鍵技術(shù)。個人云存儲作為云存儲的典型應(yīng)用之一,不僅為用戶提供了基本的數(shù)據(jù)存儲功能,還提供了文件多端同步、工作協(xié)同、多應(yīng)用匯聚等功能,已經(jīng)成為一種較為流行的數(shù)據(jù)服務(wù),但仍面臨著用戶更高服務(wù)質(zhì)量要求的挑戰(zhàn)。論文針對云存儲后端分布式文件系統(tǒng)的緩存機(jī)制開展研究,為海量用戶在海量存儲的背景下提供更高效的個人云存儲服務(wù)。通過分析真實(shí)的個人用戶訪問數(shù)據(jù)集,綜合考慮個人云存儲的應(yīng)用場景,論文發(fā)現(xiàn)傳統(tǒng)緩存機(jī)制中存在低讀寫比場景下緩存存儲空間的利用率問題,以及海量緩存文件場景下緩存替換的執(zhí)行效率問題,這些問題關(guān)系到個人云存儲服務(wù)的成本與服務(wù)質(zhì)量。針對低讀寫比場景下緩存存儲空間的利用率問題,論文提出了基于動態(tài)決策的緩存文件選擇策略。該策略通過對頻繁訪問文件的大小和類型進(jìn)行在線分析,動態(tài)判斷請求文件是否應(yīng)當(dāng)放入緩存,避免將“只一次訪問”的文件放入緩存,并通過文件訪問次數(shù)和文件訪問時間間隔對不滿足文件大小、類型的頻繁文件進(jìn)行彌補(bǔ),降低項(xiàng)誤判的可能性。實(shí)驗(yàn)證明,基于動態(tài)決策的緩存文件選擇策略可以顯著提高緩存空間的利用率,在維持緩存容量的低成本條件下,提升緩存命中率,且決策時間較低。針對海量緩存文件場景下緩存替換的執(zhí)行效率問題,論文提出了基于分代技術(shù)的緩存文件更新策略。該策略被設(shè)計實(shí)現(xiàn)為一種功能框架,可應(yīng)用不同的緩存替換算法,在保持原有緩存替換策略特性的同時,為緩存對象增加生命周期管理功能,并通過更新緩存對象的生命周期,縮小緩存替換執(zhí)行的范圍。實(shí)驗(yàn)證明,基于分代技術(shù)的緩存文件更新策略可以有效提高緩存替換效率,解決“緩存污染”問題,且靈活可配,適合用戶使用場景多樣化的個人云存儲系統(tǒng)。最后,論文在為開源分布式文件系統(tǒng)HDFS實(shí)現(xiàn)緩存模塊的基礎(chǔ)上,將以上兩種緩存策略應(yīng)用到HDFS中,從而優(yōu)化個人云存儲系統(tǒng)使用HDFS作為后端存儲時的系統(tǒng)性能,并在實(shí)際運(yùn)行中取得了較好的效果。
[Abstract]:As an important part of cloud computing, cloud storage service is the key technology to solve the problem of mass users accessing mass data. As one of the typical applications of cloud storage, personal cloud storage not only provides users with basic data storage functions, It also provides functions such as file multi-terminal synchronization, work cooperation, multi-application convergence, etc., and has become a more popular data service. However, it is still facing the challenge of higher quality of service (QoS) of users. This paper focuses on the cache mechanism of cloud storage back-end distributed file system. To provide a more efficient personal cloud storage service for mass users in the context of mass storage. By analyzing the real personal user access data set, considering the application scenarios of personal cloud storage, In this paper, we find that there are some problems in the traditional cache mechanism, such as the utilization of cache storage space in the low read-write ratio scenario, and the efficiency of cache replacement in the massive cache file scenario. These problems relate to the cost and quality of service of personal cloud storage. This paper proposes a cache file selection strategy based on dynamic decision, which dynamically determines whether the request file should be put into the cache by analyzing the size and type of frequently accessed files online. Avoid "only once access" files into the cache, and through the number of file visits and file access time interval to not meet the file size, type of frequent files to make up for, reduce the possibility of misjudgment of items. The cache file selection strategy based on dynamic decision can significantly improve the utilization of cache space and improve the cache hit ratio under the condition of maintaining cache capacity at low cost. And the decision time is low. Aiming at the efficiency of cache replacement in the massive cache file scenario, this paper proposes a cache file update strategy based on generational technology, which is designed and implemented as a functional framework. Different cache replacement algorithms can be applied to maintain the characteristics of the original cache replacement strategy, while adding life cycle management function to the cache object, and by updating the life cycle of the cache object, Experiments show that the cache file update strategy based on generational technology can effectively improve the efficiency of cache replacement, solve the problem of "cache pollution", and can be configured flexibly. Personal cloud storage system suitable for users to use a variety of scenarios. Finally, based on the implementation of cache module for open source distributed file system (HDFS), the above two caching strategies are applied to HDFS. Therefore, the performance of personal cloud storage system with HDFS as back-end storage is optimized, and good results are obtained in actual operation.
【學(xué)位授予單位】：浙江大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2015
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)重要報紙文章 前1條

1 本報記者 李響　實(shí)習(xí)記者 陳海嘉;[N];計算機(jī)世界;2012年

，

本文編號：1659499