基于Hodoop的農(nóng)業(yè)博客研究與實現(xiàn)
本文選題:農(nóng)業(yè)博客 + 分布式存儲。 參考:《西北農(nóng)林科技大學》2014年碩士論文
【摘要】:在互聯(lián)網(wǎng)技術(shù)飛速發(fā)展的時代,人們獲取信息主要來源于網(wǎng)絡(luò)資源,農(nóng)業(yè)領(lǐng)域也是如此,但農(nóng)業(yè)用戶與其他用戶不同,,計算機使用知識的匱乏使農(nóng)業(yè)用戶難以從網(wǎng)絡(luò)上獲得所需信息,尤其是在當今網(wǎng)絡(luò)信息數(shù)據(jù)量不斷增加的情況下,如何從海量的復雜數(shù)據(jù)中得到所需的相關(guān)農(nóng)業(yè)信息更是具有一定的難度。因此,為廣大農(nóng)業(yè)用戶研發(fā)一個具有實際意義的互動交流平臺是十分有必要的。博客是一種熱門的用于網(wǎng)絡(luò)信息交流的互聯(lián)網(wǎng)應用平臺,然而現(xiàn)有的博客網(wǎng)站多為綜合性交流平臺且實時性不高,并不適合農(nóng)業(yè)用戶使用。針對這些問題,本研究以分布式存儲和并行計算技術(shù)為支撐,研究與實現(xiàn)了一個具有分布式、高實時和可擴展性的農(nóng)業(yè)博客。主要研究內(nèi)容及結(jié)論如下: (1)農(nóng)業(yè)博客基礎(chǔ)模型構(gòu)建。首先根據(jù)廣大農(nóng)業(yè)用戶對農(nóng)業(yè)博客的應用需求,設(shè)計了農(nóng)業(yè)博客系統(tǒng)的整體系統(tǒng)架構(gòu)。然后針對其整體架構(gòu)改進了現(xiàn)有的博客開發(fā)模式,構(gòu)建了農(nóng)業(yè)博客的分布式存儲模型及數(shù)據(jù)并行處理模型,并對傳統(tǒng)的推-拉模式進行了改進。 (2)農(nóng)業(yè)博客系統(tǒng)設(shè)計與實現(xiàn)。首先根據(jù)系統(tǒng)的功能需求、服務體系以及基本的模型規(guī)劃了農(nóng)業(yè)博客系統(tǒng)的總體功能架構(gòu),設(shè)計了總體數(shù)據(jù)流,并對系統(tǒng)的主要功能模塊進行詳細的設(shè)計,同時對主要的數(shù)據(jù)處理過程(存儲與訪問)進行了設(shè)計說明。然后對數(shù)據(jù)庫模式進行了詳細的設(shè)計與說明。最后利用MapReduce實現(xiàn)了數(shù)據(jù)的并行處理,同時對Coprocessor數(shù)據(jù)處理與數(shù)據(jù)訪問服務的實現(xiàn)進行了具體的說明。 (3)原型測試與分析。首先根據(jù)Hadoop的框架部署了分布式測試集群并整合了MyEclipse開發(fā)環(huán)境,然后對農(nóng)業(yè)博客系統(tǒng)的用戶界面進行了系統(tǒng)功能的測試,最后在模擬大數(shù)據(jù)集的條件下完成了對系統(tǒng)的性能測試分析。原型測試結(jié)果表明系統(tǒng)具有可行性和高效性。
[Abstract]:In the era of rapid development of Internet technology, people obtain information mainly from the network resources, and so is the agricultural field, but agricultural users are different from other users.The lack of knowledge about the use of computers makes it difficult for agricultural users to obtain the required information from the network, especially when the amount of data on the network is increasing.It is more difficult to get the relevant agricultural information from the massive complex data.Therefore, it is necessary to develop a practical interactive communication platform for agricultural users.Blog is a popular Internet application platform for the exchange of information on the Internet. However, most of the existing blog sites are comprehensive communication platforms and the real-time performance is not high, so it is not suitable for agricultural users.To solve these problems, a distributed, real-time and scalable agricultural blog is studied and implemented based on distributed storage and parallel computing technology.The main contents and conclusions are as follows:1) constructing the basic model of agriculture blog.Firstly, according to the application demand of agricultural blog, the whole system structure of agricultural blog system is designed.Then, the existing blog development model is improved according to its overall architecture, the distributed storage model and data parallel processing model of agricultural blog are constructed, and the traditional push-pull model is improved.Agricultural blog system design and implementation.Firstly, according to the functional requirements of the system, the service system and the basic model, the overall functional framework of the agricultural blog system is planned, the overall data flow is designed, and the main functional modules of the system are designed in detail.At the same time, the main data processing procedures (storage and access) are described.Then the database schema is designed and explained in detail.Finally, the parallel processing of data is realized by using MapReduce, and the realization of Coprocessor data processing and data access service is explained in detail.Prototype testing and analysis.First, the distributed test cluster is deployed according to the framework of Hadoop and the MyEclipse development environment is integrated, then the user interface of the agricultural blog system is tested.Finally, the performance of the system is tested and analyzed under the condition of simulating big data set.The prototype test results show that the system is feasible and efficient.
【學位授予單位】:西北農(nóng)林科技大學
【學位級別】:碩士
【學位授予年份】:2014
【分類號】:TP393.092
【參考文獻】
相關(guān)期刊論文 前10條
1 商新娜;;Web應用中的海量數(shù)據(jù)訪問緩存技術(shù)[J];北京聯(lián)合大學學報(自然科學版);2007年03期
2 姜宇鳴;;海量數(shù)據(jù)存儲系統(tǒng)研究[J];電腦知識與技術(shù);2011年08期
3 趙玉萍;王爽;;基于Java的博客管理系統(tǒng)研究[J];電腦編程技巧與維護;2013年06期
4 高洪;楊慶平;黃震江;;基于Hadoop平臺的大數(shù)據(jù)分析關(guān)鍵技術(shù)標準化探討[J];信息技術(shù)與標準化;2013年05期
5 馬忠貴;葉斌;王成耀;王宗杰;涂序彥;;基于“推-拉”技術(shù)的多源復雜信息整合研究[J];計算機工程;2006年05期
6 李文逍;楊小虎;;基于分布式緩存的消息中間件存儲模型[J];計算機工程;2010年13期
7 彭秀媛;王昕;盧闖;宣鍇;;云計算在農(nóng)業(yè)領(lǐng)域的應用研究[J];農(nóng)業(yè)網(wǎng)絡(luò)信息;2011年02期
8 陸秋;程小輝;;基于MapReduce的決策樹算法并行化[J];計算機應用;2012年09期
9 董繼光;陳衛(wèi)衛(wèi);田浪軍;吳海佳;;大規(guī)模云存儲系統(tǒng)副本布局研究[J];計算機應用;2012年03期
10 李晶;王亮;;淺談網(wǎng)絡(luò)化應用對圖書館讀者互動及知識服務的影響[J];內(nèi)蒙古科技與經(jīng)濟;2013年01期
相關(guān)博士學位論文 前1條
1 楊國才;農(nóng)業(yè)農(nóng)村信息化云服務平臺集成關(guān)鍵技術(shù)研究[D];西南大學;2012年
本文編號:1747313
本文鏈接:http://sikaile.net/guanlilunwen/ydhl/1747313.html