基于Hadoop的CDN-P2P系統(tǒng)中內(nèi)容預(yù)測機制研究與實現(xiàn)
本文選題:CDN-P2P + 需求預(yù)測 ; 參考:《北京郵電大學》2013年碩士論文
【摘要】:近十幾年來,隨著互聯(lián)網(wǎng)的飛速發(fā)展,網(wǎng)絡(luò)信息量和用戶數(shù)急劇增長,網(wǎng)絡(luò)共享和傳輸?shù)膬?nèi)容也由簡單的文字、圖片擴展到音頻、視頻等結(jié)構(gòu)復雜、形式多樣的多媒體。為了高效進行網(wǎng)絡(luò)內(nèi)容分發(fā),緩解網(wǎng)絡(luò)擁塞,提升用戶體驗,CDN和P2P技術(shù)作為網(wǎng)絡(luò)內(nèi)容分發(fā)的主要技術(shù),在眾多領(lǐng)域被廣泛應(yīng)用?紤]到CDN和P2P技術(shù)在提供服務(wù)時與生俱有的互補性,CDN-P2P融合技術(shù)也成為新的研究熱點。 網(wǎng)絡(luò)規(guī)模的不斷擴大,共享資源信息的激增,給CDN-P2P網(wǎng)絡(luò)中節(jié)點文件共享以及邊緣服務(wù)器文件服務(wù)的提供,帶來了諸多問題。主要表現(xiàn)為:對邊緣服務(wù)器存儲負載能力以及P2P節(jié)點文件請求響應(yīng)時間的要求。CDN-P2P網(wǎng)絡(luò)需要服務(wù)的節(jié)點數(shù)和提供的文件數(shù)量不斷增大,需要在邊緣服務(wù)器和內(nèi)容源服務(wù)器之間,或者邊緣服務(wù)器之間頻繁傳送文件,不僅增加節(jié)點文件請求的響應(yīng)時間,而且消耗帶寬資源。同時,節(jié)點用戶也需要花費大量時間在海量資源信息中尋找自己需要的內(nèi)容。 改進CDN-P2P網(wǎng)絡(luò)中邊緣服務(wù)器的內(nèi)容緩存放置策略,快速響應(yīng)節(jié)點文件請求,提高節(jié)點用戶在海量信息中發(fā)現(xiàn)所需求內(nèi)容以及共享內(nèi)容的效率,是未來CDN-P2P技術(shù)重要的發(fā)展方向。本文針對上述問題,通過分析CDN-P2P網(wǎng)絡(luò)的特點,特別是節(jié)點用戶能動性參與的影響因素,融合智能推薦、搜索引擎技術(shù),對基于Hadoop的CDN-P2P原型系統(tǒng)進行了改進。 本文的研究內(nèi)容包括以下幾個方面: (1)通過分析共享內(nèi)容的類型屬性和節(jié)點需求的聯(lián)系,計算用戶偏好因子,然后結(jié)合節(jié)點用戶歷史評分相似性和偏好因子,改進協(xié)同過濾方法的預(yù)測函數(shù),對節(jié)點用戶需求預(yù)測模型進行分析研究。 (2)研究傳統(tǒng)CDN技術(shù),并結(jié)合現(xiàn)有CDN-P2P系統(tǒng)中節(jié)點子網(wǎng)組織的特性以及節(jié)點之間的相似性,對目前系統(tǒng)中內(nèi)容預(yù)存策略進行重新設(shè)計。 (3)鑒于節(jié)點用戶對內(nèi)容共享的需要,為了方便用戶查找相關(guān)信息,基于Solr設(shè)計實現(xiàn)了一個共享內(nèi)容搜索子系統(tǒng),用戶可以通過輸入關(guān)鍵詞來查找資源信息。 (4)在CDN-P2P原型系統(tǒng)中對上面提出的節(jié)點用戶需求預(yù)測模型和邊緣服務(wù)器內(nèi)容預(yù)存策略予以實現(xiàn)。
[Abstract]:In the past decade, with the rapid development of the Internet, the amount of network information and the number of users have increased dramatically. The content of network sharing and transmission has also expanded from simple text, pictures to audio, video and other complex structures, various forms of multimedia. In order to efficiently distribute network content, alleviate network congestion and enhance user experience, CDN and P2P technology are widely used in many fields as the main technology of network content distribution. Considering that CDN and P2P technologies are complementary to each other in providing services, CDN-P2P convergence technology has also become a new research hotspot. With the continuous expansion of network scale and the proliferation of shared resource information, many problems have been brought to the sharing of node files in CDN-P2P network and the provision of edge server file services. It is shown that the number of nodes and the number of files provided by the CDN-P2P network need to be increased, which is between the edge server and the content source server, and the demand for the storage load of the edge server and the response time of the file request of the P2P node. Or the frequent transfer of files between edge servers not only increases the response time of node file requests, but also consumes bandwidth resources. At the same time, node users also need to spend a lot of time searching for their own content in the massive resource information. It is an important development direction of CDN-P2P technology in the future to improve the content cache policy of edge server in CDN-P2P network, respond to the request of node file quickly, and improve the efficiency of node users finding the required content and sharing content in the massive information. In order to solve the above problems, this paper analyzes the characteristics of CDN-P2P network, especially the influence factors of node user's active participation, integrates intelligent recommendation and search engine technology, and improves the CDN-P2P prototype system based on Hadoop. The research content of this paper includes the following aspects: 1) by analyzing the relationship between the type attributes of shared content and node demand, the user preference factor is calculated, and then the prediction function of collaborative filtering method is improved by combining the similarity of node users' history score and preference factor. The node user demand prediction model is analyzed and studied. (2) the traditional CDN technology is studied, and combining with the characteristics of node subnet organization and the similarity between nodes in the existing CDN-P2P system, the content storage strategy in the current system is redesigned. In view of the needs of node users for content sharing, in order to facilitate users to find relevant information, a shared content search subsystem is designed and implemented based on Solr. Users can search resource information by entering keywords. 4) implement the node user demand prediction model and the edge server content storage strategy in the CDN-P2P prototype system.
【學位授予單位】:北京郵電大學
【學位級別】:碩士
【學位授予年份】:2013
【分類號】:TP393.02
【參考文獻】
相關(guān)期刊論文 前10條
1 方娟;梁文燦;;一種基于協(xié)同過濾的網(wǎng)格門戶推薦模型[J];電子與信息學報;2010年07期
2 徐風苓;孟祥武;王立才;;基于移動用戶上下文相似度的協(xié)同過濾推薦算法[J];電子與信息學報;2011年11期
3 黃武漢;孟祥武;王立才;;移動通信網(wǎng)中基于用戶社會化關(guān)系挖掘的協(xié)同過濾算法[J];電子與信息學報;2011年12期
4 宗瑜;金萍;陳恩紅;李紅;劉仁金;;面向Weblog的模糊協(xié)同聚類算法[J];電子與信息學報;2012年03期
5 蔣海;李軍;李忠誠;;混合內(nèi)容分發(fā)網(wǎng)絡(luò)及其性能分析模型[J];計算機學報;2009年03期
6 楊傳棟,余鎮(zhèn)危,王行剛;結(jié)合CDN與P2P技術(shù)的混合流媒體系統(tǒng)研究[J];計算機應(yīng)用;2005年09期
7 曾春,邢春曉,周立柱;個性化服務(wù)技術(shù)綜述[J];軟件學報;2002年10期
8 許海玲;吳瀟;李曉東;閻保平;;互聯(lián)網(wǎng)推薦系統(tǒng)比較研究[J];軟件學報;2009年02期
9 黃永生;孟祥武;張玉潔;;基于社會網(wǎng)絡(luò)特征的P2P內(nèi)容定位策略[J];軟件學報;2010年10期
10 陳勇;孫世新;周益民;李軍;馮永政;;基于P2P的CDN新型網(wǎng)絡(luò)及緩存替換算法[J];微電子學與計算機;2008年09期
相關(guān)博士學位論文 前1條
1 黃永生;基于用戶社會屬性的點對點內(nèi)容分發(fā)網(wǎng)絡(luò)模型研究[D];北京郵電大學;2010年
相關(guān)碩士學位論文 前3條
1 連蒴;基于Web搜索引擎系統(tǒng)的設(shè)計與實現(xiàn)[D];復旦大學;2011年
2 韓立寶;基于P2POverCDN和RTSP的流媒體代理服務(wù)器的設(shè)計與實現(xiàn)[D];西安電子科技大學;2008年
3 朱濤;基于P2P的內(nèi)容分發(fā)網(wǎng)絡(luò)的系統(tǒng)結(jié)構(gòu)資源搜索與路由算法研究[D];電子科技大學;2008年
,本文編號:1788294
本文鏈接:http://sikaile.net/kejilunwen/sousuoyinqinglunwen/1788294.html