本文選題：農(nóng)業(yè)信息化 + 農(nóng)業(yè)專家系統(tǒng)　； 參考：《云南師范大學(xué)》2017年碩士論文

【摘要】：數(shù)據(jù)是人類發(fā)展最終的智慧沉淀,是知識的集散地,是人們在自然活動(dòng)中對自然規(guī)律的探索以及研究的結(jié)果集合。大數(shù)據(jù)是指大量的數(shù)據(jù)進(jìn)行集中的結(jié)果集合。現(xiàn)在一般指的是用IT以及相關(guān)的信息化手段把傳統(tǒng)的人們對自然探索得出的知識與智慧的沉淀進(jìn)行集合,然后利用計(jì)算機(jī)的高速計(jì)算能力最終作用于我們的生活。在如今信息爆炸的今天,大數(shù)據(jù)的價(jià)值已經(jīng)被越來越多的人關(guān)注。越來越多的行業(yè)將在大數(shù)據(jù)這一板塊進(jìn)行嫁接。對于農(nóng)業(yè)來說,它的發(fā)展,到了今天已經(jīng)產(chǎn)生了極其多的經(jīng)驗(yàn)與智慧。如果把這些經(jīng)驗(yàn)與智慧化為大數(shù)據(jù),那么對整個(gè)農(nóng)業(yè)未來的發(fā)展將是極其有利的。于是,農(nóng)業(yè)大數(shù)據(jù)聚合平臺的研究將是極其有價(jià)值的。但是,由于互聯(lián)網(wǎng)上的數(shù)據(jù)量過于龐大,種類極其多,很多數(shù)據(jù)都是無效數(shù)據(jù)。因此,本文對智能農(nóng)業(yè)大數(shù)據(jù)聚合的相關(guān)理論與技術(shù)進(jìn)行研究,提出了一種快速獲取有效農(nóng)業(yè)大數(shù)據(jù)的架構(gòu)與機(jī)制,設(shè)計(jì)并實(shí)現(xiàn)了基于規(guī)則的智能農(nóng)業(yè)大數(shù)據(jù)聚合平臺。本文的主要研究內(nèi)容及研究成果如下:(1)設(shè)計(jì)了高性能網(wǎng)絡(luò)蜘蛛。考慮到互聯(lián)網(wǎng)數(shù)據(jù)量極其巨大,為了能夠高效獲取有效的農(nóng)業(yè)數(shù)據(jù),本文基于多任務(wù)并發(fā)機(jī)制,設(shè)計(jì)了一個(gè)高性能的網(wǎng)絡(luò)蜘蛛。在此過程中由于要頻繁進(jìn)行大數(shù)據(jù)的存儲,會造成內(nèi)存碎片,降低操作系統(tǒng)性能,所以基于內(nèi)存的智能垃圾回收機(jī)制設(shè)計(jì)了智能農(nóng)業(yè)大數(shù)據(jù)聚合平臺的內(nèi)存池與數(shù)據(jù)通用存儲對象。整個(gè)高性能網(wǎng)絡(luò)蜘蛛的核心組件包括:內(nèi)存池、通用存儲對象、配置管理器、URI標(biāo)準(zhǔn)化模塊、數(shù)據(jù)獲取器、基于規(guī)則的智能推理過濾器、URL識別器、爬行隊(duì)列、編碼轉(zhuǎn)換器、特殊字符過濾器等11個(gè)子模塊。(2)構(gòu)建了基于規(guī)則的智能農(nóng)業(yè)大數(shù)據(jù)聚合平臺。由于互聯(lián)網(wǎng)數(shù)據(jù)種類繁多、龐雜,本文設(shè)計(jì)了基于規(guī)則的農(nóng)業(yè)專家系統(tǒng)來進(jìn)行數(shù)據(jù)有效性判斷,精準(zhǔn)的把互聯(lián)網(wǎng)中農(nóng)業(yè)相關(guān)的數(shù)據(jù)進(jìn)行聚合,把農(nóng)業(yè)無關(guān)的無效數(shù)據(jù)進(jìn)行排除。整個(gè)專家系統(tǒng)基于農(nóng)業(yè)特征知識庫構(gòu)建,基于知識庫中的農(nóng)業(yè)特征與相關(guān)的邏輯關(guān)系,在高性能網(wǎng)絡(luò)蜘蛛進(jìn)行數(shù)據(jù)獲取的基礎(chǔ)上通過它來進(jìn)行智能推理過濾。最終,實(shí)現(xiàn)了一個(gè)在海量的、龐雜的互聯(lián)網(wǎng)的數(shù)據(jù)中能有效聚合農(nóng)業(yè)相關(guān)數(shù)據(jù)的農(nóng)業(yè)大數(shù)據(jù)聚合平臺。
[Abstract]:Data is the final wisdom precipitation of human development, is the distribution of knowledge, is the people in natural activities in the exploration of natural laws and the results of the collection of research.Big data refers to a large number of data for the set of results.At present, it is generally referred to the collection of traditional knowledge and wisdom from natural exploration by IT and related information methods, and then the use of the high speed computing power of the computer to act on our lives.In today's information explosion, big data's value has been paid more and more attention.More and more industries will be in big data this plate graft.For agriculture, its development, to today has produced a great deal of experience and wisdom.If these experiences and wisdom become big data, then the future development of the whole agriculture will be extremely beneficial.Therefore, the agricultural big data aggregation platform research will be extremely valuable.However, due to the huge amount of data on the Internet, a lot of data are invalid.Therefore, this paper studies the relevant theory and technology of big data aggregation of intelligent agriculture, proposes a framework and mechanism for obtaining effective agricultural big data quickly, and designs and implements a rule-based intelligent agricultural big data aggregation platform.The main contents and results of this paper are as follows: 1) A high performance web spider is designed.Considering the huge amount of Internet data, in order to obtain effective agricultural data efficiently, a high performance web spider is designed based on multi-task concurrency mechanism.In this process, due to the frequent storage of big data, memory fragmentation will be caused, and the performance of the operating system will be reduced.Therefore, the memory storage pool and data universal storage object of big data aggregation platform are designed based on the memory intelligent garbage collection mechanism.The core components of the whole high-performance web spider include: memory pool, universal storage object, configuration manager Uri standardization module, data acquirer, intelligent reasoning filter based on rules, URL recognizer, crawling queue, encoding converter, etc.Special character filter and other 11 submodules. 2) A rule-based intelligent agricultural big data aggregation platform is constructed.Due to the wide variety of Internet data, this paper designed a rule-based agricultural expert system to judge the validity of the data, accurately aggregate the agri-related data in the Internet, and eliminate the agnostic invalid data.The whole expert system is constructed based on the agricultural feature knowledge base, and based on the agricultural features and related logic relations in the knowledge base, the intelligent reasoning filtering is carried out through the high performance web spider on the basis of its data acquisition.Finally, an agri- aggregation platform which can effectively aggregate agri-related data in a large amount of Internet data is implemented.
【學(xué)位授予單位】：云南師范大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP311.13;S126

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 王通;劉放;;大數(shù)據(jù) 新農(nóng)業(yè)革命[J];農(nóng)經(jīng);2016年01期

2 袁曉慶;;“互聯(lián)網(wǎng)+”農(nóng)業(yè):助推農(nóng)業(yè)走進(jìn)4.0時(shí)代[J];互聯(lián)網(wǎng)經(jīng)濟(jì);2015年Z2期

3 羅文華;王強(qiáng);;Windows環(huán)境下基于棧和堆的應(yīng)用程序行為解析方法研究[J];警察技術(shù);2015年01期

4 郭承坤;劉延忠;陳英義;孫敏;屠星月;;發(fā)展農(nóng)業(yè)大數(shù)據(jù)的主要問題及主要任務(wù)[J];安徽農(nóng)業(yè)科學(xué);2014年27期

5 寇曉斌;楊琴;王亮亮;;主流處理器體系結(jié)構(gòu)與架構(gòu)發(fā)展現(xiàn)狀綜述[J];微型機(jī)與應(yīng)用;2014年16期

6 王德超;;C/C++編譯系統(tǒng)內(nèi)存分配分類比較探究[J];軟件;2014年02期

7 Shengmei Luo;Zhikun Wang;Zhiping Wang;;Big-Data Analytics:Challenges,Key Technologies and Prospects[J];ZTE Communications;2013年02期

8 維克托·邁爾·舍恩伯格;周濤;;大數(shù)據(jù)時(shí)代 生活、工作與思維的大變革[J];人力資源管理;2013年03期

9 許健;于鴻洋;;一種Linux多線程應(yīng)用下內(nèi)存池的設(shè)計(jì)與實(shí)現(xiàn)[J];電子技術(shù)應(yīng)用;2012年11期

10 周仕來;;頂板防治措施專家系統(tǒng)知識庫的構(gòu)建[J];煤礦現(xiàn)代化;2012年05期

相關(guān)碩士學(xué)位論文 前6條

1 孝瑞;內(nèi)存動(dòng)態(tài)安全監(jiān)測及防范研究[D];華北電力大學(xué);2015年

2 趙珍;視頻會議中緩沖隊(duì)列的設(shè)計(jì)與實(shí)現(xiàn)[D];西安電子科技大學(xué);2012年

3 范昕昕;我國農(nóng)業(yè)信息化測評及發(fā)展戰(zhàn)略研究[D];中國海洋大學(xué);2010年

4 王治江;面向領(lǐng)域的垂直搜索系統(tǒng)研究與實(shí)現(xiàn)[D];大連理工大學(xué);2009年

5 李江雄;嵌入式linux內(nèi)存管理設(shè)計(jì)與實(shí)現(xiàn)[D];華中科技大學(xué);2008年

6 王炬;農(nóng)業(yè)信息化進(jìn)程中的問題及對策研究[D];西南農(nóng)業(yè)大學(xué);2003年

，

本文編號：1768552