本文選題：分布式文件系統(tǒng)　切入點(diǎn)：圖片服務(wù)器　出處：《北方工業(yè)大學(xué)》2017年碩士論文

【摘要】：隨著互聯(lián)網(wǎng)技術(shù)的蓬勃發(fā)展,一些大型的社交網(wǎng)站如Facebook、騰訊,大型的電商平臺(tái)如淘寶、京東,對(duì)于圖片服務(wù)的需求也越來(lái)越大,這些大型的網(wǎng)站需要對(duì)千億級(jí)別數(shù)量的圖片進(jìn)行存儲(chǔ)和管理。同時(shí)移動(dòng)終端深入人們的生活,移動(dòng)社交App得到了廣泛使用。在移動(dòng)社交APP中往往也會(huì)使用大量的圖片資源,如微信朋友圈、Instagram的圖片分享等。如何支持如此大規(guī)模的圖片存儲(chǔ)和處理成了一個(gè)亟待解決的問(wèn)題。在App中瀏覽圖片會(huì)消耗較多的網(wǎng)絡(luò)流量,影響加載速度,因此,根據(jù)移動(dòng)端的特點(diǎn),大部分App采用首先顯示縮略圖,根據(jù)用戶(hù)需求再加載原圖的策略。在圖片服務(wù)器設(shè)計(jì)和實(shí)現(xiàn)過(guò)程中采用緩存技術(shù)來(lái)加快縮略圖的生成速度,減少磁盤(pán)I/O。但是當(dāng)前的緩存機(jī)制更多關(guān)注的是緩存的訪問(wèn)頻率、最近訪問(wèn)時(shí)間等因素,并沒(méi)有過(guò)多關(guān)注數(shù)據(jù)生成用戶(hù)之間的社交關(guān)系,也沒(méi)有考慮移動(dòng)用戶(hù)對(duì)縮略圖和原圖的不同訪問(wèn)模式。因此,本文把社交網(wǎng)絡(luò)用戶(hù)中的影響度同緩存對(duì)象緊密聯(lián)系,同時(shí)把原圖的緩存對(duì)象和縮略圖的緩存對(duì)象相關(guān)聯(lián)。論文的主要工作包括以下四個(gè)方面:第一,本文首先通過(guò)對(duì)縮略圖和原圖的分析,把緩存區(qū)劃分為兩個(gè)部分:縮略圖緩存區(qū)和原圖緩存區(qū)。然后分析了緩存區(qū)中的數(shù)據(jù)特性,定義了不同的緩存對(duì)象模型。第二,對(duì)于不同的緩存區(qū)定義了不同的緩存價(jià)值的公式,并對(duì)緩存價(jià)值的計(jì)算公式進(jìn)行詳細(xì)介紹。在定義緩存價(jià)值公式的同時(shí),也定義了社交網(wǎng)絡(luò)中用戶(hù)影響度的計(jì)算方法、LRU價(jià)值的計(jì)算方法。結(jié)合緩存對(duì)象模型提出了基于社交關(guān)系的圖片緩存替換算法。在傳統(tǒng)緩存替換算法的基礎(chǔ)上增加用戶(hù)的社交關(guān)系以及縮略圖和原圖的關(guān)聯(lián)關(guān)系,通過(guò)計(jì)算圖片的緩存價(jià)值進(jìn)行緩存替換。第三,結(jié)合項(xiàng)目背景,設(shè)計(jì)并實(shí)現(xiàn)了圖片服務(wù)器。本文設(shè)計(jì)的圖片服務(wù)器向客戶(hù)端提供了簡(jiǎn)單易用的接口,能夠很方便的應(yīng)用到網(wǎng)站的圖片存儲(chǔ)系統(tǒng)中。第四,在本文的最后通過(guò)設(shè)計(jì)合理的模擬性實(shí)驗(yàn)加以驗(yàn)證,并對(duì)實(shí)驗(yàn)的結(jié)果進(jìn)行分析和總結(jié)。實(shí)驗(yàn)表明,本文提出的緩存替換替換策略可以提高命中率。
[Abstract]:With the rapid development of Internet technology, some large social networking sites such as Facebook, Tencent, large e-commerce platforms such as Taobao, JingDong, the demand for photo services is also growing.These large websites need to store and manage hundreds of billions of images.At the same time, mobile terminals deeply into people's lives, mobile social App has been widely used.Mobile social APP also often uses a large number of photo resources, such as the WeChat Moments and Instagram photo sharing.How to support such a large-scale picture storage and processing has become a problem to be solved.Browsing pictures in App consumes more network traffic and affects loading speed. Therefore, according to the characteristics of mobile side, most of App adopts the strategy of displaying thumbnail first and loading original image according to user demand.In the process of design and implementation of picture server, cache technology is used to speed up the generation of thumbnail and reduce disk I / O.However, the current caching mechanism pays more attention to the frequency of cache access, recent access time and other factors, and does not pay much attention to the social relationship between data generation users.The different access modes of thumbnails and original images by mobile users are not taken into account.Therefore, the influence degree of social network users is closely related to the cache object, and the cache object of the original image and the cache object of the thumbnail are associated with each other.The main work of this paper includes the following four aspects: firstly, through the analysis of thumbnail and original image, the buffer is divided into two parts: thumbnail cache and original image cache.Then, the data characteristics in cache are analyzed, and different cache object models are defined.Secondly, different cache value formulas are defined for different cache areas, and the calculation formulas of cache value are introduced in detail.In addition to defining the cache value formula, this paper also defines the calculation method of user influence degree in social network and the calculation method of LRU value.A new image cache replacement algorithm based on social relation is proposed based on cache object model.Based on the traditional cache replacement algorithm, the social relationship of users and the correlation between thumbnail and original image are added, and the cache replacement is carried out by calculating the cache value of images.Thirdly, the picture server is designed and implemented according to the background of the project.The picture server designed in this paper provides the client with a simple and easy to use interface, which can be easily applied to the image storage system of the website.At the end of this paper, we design a reasonable simulation experiment to verify it, and analyze and summarize the results of the experiment.Experiments show that the cache replacement strategy proposed in this paper can improve the hit rate.
【學(xué)位授予單位】：北方工業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類(lèi)號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 邢起源;王菁;閆阿賓;韓燕波;;一種基于社交關(guān)系的移動(dòng)緩存替換算法[J];計(jì)算機(jī)科學(xué);2016年06期

2 吳靜;戴志濤;;FastDFS針對(duì)數(shù)據(jù)采集應(yīng)用的改進(jìn)[J];軟件;2014年12期

3 王文博;王菁;邢起源;韓燕波;;移動(dòng)即時(shí)通訊軟件的緩存替換策略[J];計(jì)算機(jī)科學(xué)與探索;2015年03期

4 周江;王偉平;孟丹;馬燦;古曉艷;蔣杰;;面向大數(shù)據(jù)分析的分布式文件系統(tǒng)關(guān)鍵技術(shù)[J];計(jì)算機(jī)研究與發(fā)展;2014年02期

5 廖彬;于炯;張?zhí)?楊興耀;;基于分布式文件系統(tǒng)HDFS的節(jié)能算法[J];計(jì)算機(jī)學(xué)報(bào);2013年05期

6 張文盛;鄭漢華;;基于MongoDB構(gòu)建高性能網(wǎng)站技術(shù)研究[J];吉林師范大學(xué)學(xué)報(bào)(自然科學(xué)版);2013年01期

7 林榮恒;章暉;鄒華;;面向SNS用戶(hù)訪問(wèn)行為的Web緩存預(yù)測(cè)替換[J];北京郵電大學(xué)學(xué)報(bào);2012年01期

8 喬秀全;楊春;李曉峰;陳俊亮;;社交網(wǎng)絡(luò)服務(wù)中一種基于用戶(hù)上下文的信任度計(jì)算方法[J];計(jì)算機(jī)學(xué)報(bào);2011年12期

9 白文江;;基于Android平臺(tái)的移動(dòng)應(yīng)用開(kāi)發(fā)研究[J];太原大學(xué)學(xué)報(bào);2011年03期

10 薛媛媛;;分布式服務(wù)器負(fù)載均衡分析與實(shí)現(xiàn)[J];青海大學(xué)學(xué)報(bào)(自然科學(xué)版);2011年03期

相關(guān)碩士學(xué)位論文 前7條

1 邢起源;移動(dòng)云環(huán)境中基于社交關(guān)系的緩存技術(shù)研究[D];北方工業(yè)大學(xué);2016年

2 郭建國(guó);基于分布式文件系統(tǒng)FastDFS的圖片服務(wù)器件設(shè)計(jì)與應(yīng)用[D];中國(guó)科學(xué)院大學(xué)(工程管理與信息技術(shù)學(xué)院);2015年

3 王春;高性能分布式圖片服務(wù)器研究與實(shí)現(xiàn)[D];上海交通大學(xué);2015年

4 鄒紹武;Android移動(dòng)應(yīng)用開(kāi)發(fā)中數(shù)據(jù)流量控制技術(shù)的研究應(yīng)用[D];內(nèi)蒙古師范大學(xué);2014年

5 韓增曦;分布式文件系統(tǒng)FastDFS的研究與應(yīng)用[D];大連理工大學(xué);2014年

6 張軼彬;分布式文件系統(tǒng)客戶(hù)端的設(shè)計(jì)與實(shí)現(xiàn)[D];上海交通大學(xué);2011年

7 楊�？�;多媒體資源緩存服務(wù)器的設(shè)計(jì)與實(shí)現(xiàn)[D];北京郵電大學(xué);2010年

，

本文編號(hào)：1728198