【摘要】：在信息技術(shù)高速發(fā)展的時代,信息過載現(xiàn)象越發(fā)嚴(yán)重,如何能在大量的資源中快速挖掘出用戶感興趣的信息,已成為亟待解決的問題,在這種時代背景下,推薦系統(tǒng)應(yīng)運而生。然而在實際應(yīng)用中,稀疏矩陣問題是致使推薦系統(tǒng)推薦準(zhǔn)確率下降的一個重要原因。另外,用戶的行為數(shù)據(jù)呈爆炸式增長,這種現(xiàn)象導(dǎo)致單臺服務(wù)器已經(jīng)很難滿足推薦系統(tǒng)海量數(shù)據(jù)的運算需要。綜上所述,基于Hadoop云平臺的推薦系統(tǒng)的研究具有理論和實際的雙重價值。協(xié)同過濾推薦系統(tǒng)是被使用最廣泛的推薦系統(tǒng),因此本文以協(xié)同過濾推薦系統(tǒng)為主要研究目標(biāo),旨在解決推薦系統(tǒng)的稀疏矩陣、處理海量數(shù)據(jù)計算瓶頸等問題。基于以上兩種關(guān)鍵問題,本文從算法與系統(tǒng)兩個層面進(jìn)行優(yōu)化,研究并設(shè)計了一種基于Hadoop云平臺的推薦系統(tǒng)。本文工作主要包括以下幾點內(nèi)容:1)閱讀了大量有關(guān)推薦系統(tǒng)協(xié)同過濾算法的文獻(xiàn),總結(jié)前人的研究成果和現(xiàn)在國內(nèi)外相關(guān)研究狀況。2)為了有效防止傳統(tǒng)協(xié)同過濾方法存在的項目維度過高、數(shù)據(jù)稀疏性、主觀因子干擾等問題,本文提出了一種基于用戶興趣模型以及懲罰主觀因子的協(xié)同過濾算法(Interests Model Weaken S.ubjective Collaborative Filtering,IMWS-CF)。該方法引入興趣因子,用戶興趣評分因子、懲罰主觀因子等概念,通過采用高效可行的方法來降低數(shù)據(jù)集的稀疏性與提高算法的精度,進(jìn)而解決推薦系統(tǒng)的稀疏矩陣問題。3)在研究了推薦系統(tǒng)技術(shù)細(xì)節(jié)的基礎(chǔ)上,利用之前的優(yōu)化算法(IMWS-CF),設(shè)計一種基于Hadoop云平臺的推薦系統(tǒng)。運用模塊化的思想對系統(tǒng)進(jìn)行優(yōu)化設(shè)計,在考慮高并發(fā)、穩(wěn)定性、易擴(kuò)展性等因素的同時,還提出并設(shè)計了環(huán)境分析引擎,基于不同的推薦環(huán)境,采用不同的推薦策略,從系統(tǒng)架構(gòu)層面上優(yōu)化了推薦系統(tǒng)的精確性。4)從稀疏矩陣與并行計算能力兩方面進(jìn)行實驗設(shè)計,驗證本文設(shè)計與實現(xiàn)的基于Hadoop云平臺推薦系統(tǒng),其在緩解稀疏矩陣問題與海量計算瓶頸問題上,都起到了優(yōu)化的作用。
[Abstract]:In the era of rapid development of information technology, the phenomenon of information overload is becoming more and more serious. How to quickly excavate the information interested by users in a large number of resources has become a problem to be solved urgently. Under this background, recommendation system emerges as the times require. However, in practical application, sparse matrix problem is an important reason for the decrease of recommendation accuracy. In addition, the behavior data of users is increasing explosively, which makes it difficult for a single server to meet the need of computing massive data in recommendation system. To sum up, the research of recommendation system based on Hadoop cloud platform has both theoretical and practical value. Collaborative filtering recommendation system is the most widely used recommendation system, so this paper focuses on collaborative filtering recommendation system to solve the sparse matrix of recommendation system and deal with the bottleneck of mass data computing. Based on the above two key problems, this paper studies and designs a recommendation system based on Hadoop cloud platform by optimizing the algorithm and system. This paper mainly includes the following contents: 1) read a lot of literature about collaborative filtering algorithm of recommendation system. In order to effectively prevent the traditional collaborative filtering methods, such problems as high project dimension, data sparsity, subjective factor interference and so on, are summarized. This paper presents a collaborative filtering algorithm (Interests Model Weaken S.ubjective Collaborative Filtering,IMWS-CF based on user interest model and penalty subjective factor. In this method, the concepts of interest factor, user interest score factor and penalty subjective factor are introduced to reduce the sparsity of data sets and improve the accuracy of the algorithm by using efficient and feasible methods. On the basis of studying the technical details of the recommendation system, a recommendation system based on the Hadoop cloud platform is designed by using the previous optimization algorithm (IMWS-CF). Using modularization to optimize the design of the system, considering the factors of high concurrency, stability, expansibility and so on, the environment analysis engine is proposed and designed. Based on the different recommendation environment, different recommendation strategies are adopted. The accuracy of recommendation system is optimized from the system architecture level. 4) the sparse matrix and parallel computing ability are designed experimentally to verify the design and implementation of the recommendation system based on Hadoop cloud platform. It plays an important role in alleviating the sparse matrix problem and the bottleneck problem of mass computing.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2016
【分類號】：TP391.3;TP393.09

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 趙偉;李俊鋒;韓英;張紅濤;;Hadoop云平臺下的基于用戶協(xié)同過濾算法研究[J];計算機(jī)測量與控制;2015年06期

2 簡巖;楊道平;;Hadoop Yarn標(biāo)簽調(diào)度的應(yīng)用[J];企業(yè)技術(shù)開發(fā);2015年15期

3 趙海燕;侯景德;陳慶奎;;結(jié)合時間權(quán)重與信任關(guān)系的協(xié)同過濾推薦算法[J];計算機(jī)應(yīng)用研究;2015年12期

4 孫彥超;韓鳳霞;;基于協(xié)同過濾算法的個性化圖書推薦系統(tǒng)的研究[J];圖書館理論與實踐;2015年04期

5 江南;羅軍;;基于可信相似度的協(xié)同過濾算法[J];計算機(jī)時代;2015年04期

6 丁少衡;姬東鴻;王路路;;基于用戶屬性和評分的協(xié)同過濾推薦算法[J];計算機(jī)工程與設(shè)計;2015年02期

7 尹穎;林慶;林涵陽;;HDFS中高效存儲小文件的方法[J];計算機(jī)工程與設(shè)計;2015年02期

8 董春濤;李文婷;沈晴霓;吳中海;;Hadoop YARN大數(shù)據(jù)計算框架及其資源調(diào)度機(jī)制研究[J];信息通信技術(shù);2015年01期

9 張韓飛;;推薦系統(tǒng)綜述[J];電子世界;2014年04期

10 王彥明;奉國和;薛云;;近年來Hadoop國外研究綜述[J];計算機(jī)系統(tǒng)應(yīng)用;2013年06期

相關(guān)博士學(xué)位論文 前2條

1 尹航;信息推薦系統(tǒng)中的協(xié)同過濾技術(shù)研究[D];東北大學(xué);2012年

2 孫小華;協(xié)同過濾系統(tǒng)的稀疏性與冷啟動問題研究[D];浙江大學(xué);2005年

相關(guān)碩士學(xué)位論文 前2條

1 李鑫;Hadoop框架的擴(kuò)展和性能調(diào)優(yōu)[D];西安建筑科技大學(xué);2012年

2 王金輝;基于標(biāo)簽的協(xié)同過濾稀疏性問題研究[D];中國科學(xué)技術(shù)大學(xué);2011年

，

本文編號：2238100