本文選題：網(wǎng)站分析　切入點：MapReduce　出處：《北京郵電大學(xué)》2014年碩士論文

【摘要】：隨著互聯(lián)網(wǎng)高速發(fā)展和網(wǎng)站維護成本不斷增加,本文提出Webmon整站分析系統(tǒng)。網(wǎng)站維護者通過Webmon整站分析系統(tǒng),可以實現(xiàn)對網(wǎng)站的全面監(jiān)控和分析,保證網(wǎng)站質(zhì)量。 針對本課題定位的Webmon整站分析系統(tǒng)的目標(biāo),本文對Webmon整站分析系統(tǒng)進行了需求分析并給出了總體設(shè)計。Webmon整站分析系統(tǒng)采用Web三層架構(gòu),主要功能模塊有網(wǎng)站抓取模塊、數(shù)據(jù)分析模塊和報告投遞模塊。本文分析了各個模塊的功能需求。本文對系統(tǒng)中的關(guān)鍵技術(shù)做了調(diào)研分析,主要是MapReduce模型、分布式數(shù)據(jù)庫HBase以及PageRank算法。 本文著重從數(shù)據(jù)類型、數(shù)據(jù)訪問和數(shù)據(jù)表來完成數(shù)據(jù)庫設(shè)計,著重從模塊工作流程和關(guān)鍵技術(shù)實現(xiàn)來完成各個功能模塊的詳細設(shè)計。本文設(shè)計實現(xiàn)基于PageRank算法和基于網(wǎng)站邏輯結(jié)構(gòu)的網(wǎng)站平均性能的計算,并且得出模擬的網(wǎng)站用戶體驗得分,全方面的考慮了網(wǎng)站的性能數(shù)據(jù)。 本文對Webmon整站分析系統(tǒng)的功能點進行了功能測試,驗證了系統(tǒng)的具體實現(xiàn)與預(yù)期結(jié)果一致。本文在最后進行了整體的總結(jié)和展望,并針對系統(tǒng)中存在的不完善的地方,提出了具體的改進方向。
[Abstract]:With the rapid development of the Internet and the increasing cost of website maintenance, this paper puts forward the Webmon whole station analysis system. Website maintainers can realize the overall monitoring and analysis of the website through the Webmon whole station analysis system, and ensure the quality of the website. Aiming at the goal of Webmon whole station analysis system, this paper analyzes the requirement of Webmon whole station analysis system and gives the overall design. Webmon whole station analysis system adopts Web three-layer structure, and the main function module is web crawling module. Data analysis module and report delivery module. This paper analyzes the functional requirements of each module. In this paper, the key technologies in the system are investigated and analyzed, including MapReduce model, distributed database HBase and PageRank algorithm. This paper focuses on data types, data access and data tables to complete the database design, In this paper, we design and implement the average performance of the website based on the PageRank algorithm and the logical structure of the website. And the simulation of the site user experience score, all aspects of the performance of the site data. In this paper, the function point of Webmon whole station analysis system is tested, and the concrete realization of the system is in accordance with the expected result. In the end, the whole summary and prospect are given, and the imperfections in the system are pointed out. The improvement direction is put forward.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2014
【分類號】：TP393.092

【參考文獻】

相關(guān)期刊論文 前6條

1 黃德才;戚華春;;PageRank算法研究[J];計算機工程;2006年04期

2 李成華;張新訪;金海;向文;;MapReduce:新型的分布式并行計算編程模型[J];計算機工程與科學(xué);2011年03期

3 周立柱,林玲;聚焦爬蟲技術(shù)研究綜述[J];計算機應(yīng)用;2005年09期

4 劉巍巍,徐成,李仁發(fā);嵌入式數(shù)據(jù)庫BerkeleyDB的原理與應(yīng)用[J];科學(xué)技術(shù)與工程;2005年02期

5 曹軍;Google的PageRank技術(shù)剖析[J];情報雜志;2002年10期

6 胡少榮;孟嗣儀;劉云;張彥超;丁飛;;網(wǎng)頁信息自動抽取技術(shù)的研究[J];鐵路計算機應(yīng)用;2010年09期

，

本文編號：1687717