WEB信息整合平臺設計與實現(xiàn)
發(fā)布時間：2018-09-17 08:11
【摘要】：隨著Internet技術(shù)的高速發(fā)展,網(wǎng)絡信息資源的快速增長,網(wǎng)絡已成為人們獲取數(shù)據(jù)的重要來源。面對龐大的網(wǎng)絡資源,搜索引擎為人們檢索提供重要的技術(shù)手段。然而,傳統(tǒng)的搜索引擎是基于單詞的檢索,存在一定的局限性,如搜索結(jié)果存在大量無關的網(wǎng)頁、由于轉(zhuǎn)載而造成信息內(nèi)容雷同等。因此,極有必要對網(wǎng)絡信息資源進行整合,以幫助人們從海量網(wǎng)絡資源中,提煉出人們所關心的特定信息,并對數(shù)據(jù)重新整合與統(tǒng)一的展現(xiàn)。本文的主要研究工作就是整合WEB資源信息,使互聯(lián)網(wǎng)用戶能夠快速準確地搜尋到自己需要的信息。 首先,本文對WEB信息整合中的相關理論和技術(shù)研究,包括信息整合兩種方法、三大組成模塊以及四種關鍵技術(shù)等。并在設計過程中對各模塊涉及知識做全面綜述,包括本體概念、網(wǎng)絡爬蟲、信息抽取、資源描述框架等。 其次,本文設計并實現(xiàn)了一種WEB信息整合平臺原型系統(tǒng),該系統(tǒng)以本體為指導。設計了系統(tǒng)總體結(jié)構(gòu)框架模型,系統(tǒng)由4大模塊組成：數(shù)據(jù)采集、信息抽取、存儲模型、前臺呈現(xiàn)。提出了基于本體和搜索引擎聚焦網(wǎng)絡爬蟲,基于本體的頁面分析過濾算法,基于本體和DOM樹路徑的信息抽取規(guī)則,以及基于RDF的數(shù)據(jù)存儲模型和基于B/S前臺結(jié)果呈現(xiàn)等一系列設計方案。通過該信息整合平臺,用戶可以設置需要整合的領域信息,系統(tǒng)能夠檢索并整合出互聯(lián)網(wǎng)中相關領域資源,并將結(jié)果以統(tǒng)一的、結(jié)構(gòu)的、形象的展示給用戶。該系統(tǒng)不需要對不同數(shù)據(jù)源分別建立包裝器,而是作用域整個互聯(lián)網(wǎng)之上,能夠融合互聯(lián)網(wǎng)中多種異構(gòu)資源。 最后,本文還對WEB信息整合平臺做了綜合測試,包括爬蟲的效率與抓取量測試、數(shù)據(jù)抽取率測試等。測試證明系統(tǒng)能整合互聯(lián)網(wǎng)中部分異質(zhì)的數(shù)據(jù)源,但也存在一些不足。
[Abstract]:With the rapid development of Internet technology and the rapid growth of network information resources, the network has become an important source for people to obtain data. Facing the huge network resources, the search engine provides the important technical means for people's retrieval. However, the traditional search engine is based on word retrieval, there are some limitations, such as search results have a large number of unrelated web pages, because of reprinting and resulting in the same information content and so on. Therefore, it is very necessary to integrate the network information resources to help people extract the specific information that people care about from the massive network resources, and to reintegrate and unify the data. The main research work of this paper is to integrate WEB resource information so that Internet users can quickly and accurately search for the information they need. Firstly, this paper studies the theory and technology of WEB information integration, including two methods of information integration, three modules and four key technologies. In the process of design, the knowledge involved in each module is summarized, including ontology concept, web crawler, information extraction, resource description framework and so on. Secondly, this paper designs and implements a prototype system of WEB information integration platform, which is guided by ontology. The system is composed of four modules: data acquisition, information extraction, storage model and foreground presentation. This paper proposes a web crawler based on ontology and search engine, a page analysis filtering algorithm based on ontology, and information extraction rules based on ontology and DOM tree path. And a series of design schemes, such as data storage model based on RDF and foreground result presentation based on B / S, etc. Through the information integration platform, the user can set up the domain information that needs to be integrated. The system can retrieve and integrate the related domain resources in the Internet, and display the results to the user in a unified, structured and vivid way. The system does not need to set up wrappers for different data sources separately, but the scope of the entire Internet, and can integrate a variety of heterogeneous resources in the Internet. Finally, the paper also makes a comprehensive test on WEB information integration platform, including crawler efficiency and crawl test, data extraction rate test and so on. The test shows that the system can integrate some heterogeneous data sources in the Internet, but there are some shortcomings.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP391.1
本文編號：2245262
[Abstract]:With the rapid development of Internet technology and the rapid growth of network information resources, the network has become an important source for people to obtain data. Facing the huge network resources, the search engine provides the important technical means for people's retrieval. However, the traditional search engine is based on word retrieval, there are some limitations, such as search results have a large number of unrelated web pages, because of reprinting and resulting in the same information content and so on. Therefore, it is very necessary to integrate the network information resources to help people extract the specific information that people care about from the massive network resources, and to reintegrate and unify the data. The main research work of this paper is to integrate WEB resource information so that Internet users can quickly and accurately search for the information they need. Firstly, this paper studies the theory and technology of WEB information integration, including two methods of information integration, three modules and four key technologies. In the process of design, the knowledge involved in each module is summarized, including ontology concept, web crawler, information extraction, resource description framework and so on. Secondly, this paper designs and implements a prototype system of WEB information integration platform, which is guided by ontology. The system is composed of four modules: data acquisition, information extraction, storage model and foreground presentation. This paper proposes a web crawler based on ontology and search engine, a page analysis filtering algorithm based on ontology, and information extraction rules based on ontology and DOM tree path. And a series of design schemes, such as data storage model based on RDF and foreground result presentation based on B / S, etc. Through the information integration platform, the user can set up the domain information that needs to be integrated. The system can retrieve and integrate the related domain resources in the Internet, and display the results to the user in a unified, structured and vivid way. The system does not need to set up wrappers for different data sources separately, but the scope of the entire Internet, and can integrate a variety of heterogeneous resources in the Internet. Finally, the paper also makes a comprehensive test on WEB information integration platform, including crawler efficiency and crawl test, data extraction rate test and so on. The test shows that the system can integrate some heterogeneous data sources in the Internet, but there are some shortcomings.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP391.1
【參考文獻】
相關期刊論文 前10條
1 程文濤;師雪霖;;以本體為指導的Web網(wǎng)頁信息抽取方法[J];北京化工大學學報(自然科學版);2011年04期
2 李保利,陳玉忠,俞士汶;信息抽取研究綜述[J];計算機工程與應用;2003年10期
3 蔡俊杰;孫建伶;董金祥;;建立Web信息集成系統(tǒng)[J];計算機科學;2001年12期
4 楊先娣;彭智勇;劉君強;李旭輝;;信息集成研究綜述[J];計算機科學;2006年07期
5 周德懋;李舟軍;;高性能網(wǎng)絡爬蟲:研究綜述[J];計算機科學;2009年08期
6 鄒嘉麟,陳家訓;Web信息資源整合系統(tǒng)模型和方法[J];計算機工程;2004年12期
7 李勇;韓亮;;主題搜索引擎中網(wǎng)絡爬蟲的搜索策略研究[J];計算機工程與科學;2008年03期
8 李效東,顧毓清;基于DOM的Web信息提取[J];計算機學報;2002年05期
9 周立柱,林玲;聚焦爬蟲技術(shù)研究綜述[J];計算機應用;2005年09期
10 劉金紅;陸余良;;主題網(wǎng)絡爬蟲研究綜述[J];計算機應用研究;2007年10期
相關碩士學位論文 前5條
1 方少卿;Web就業(yè)信息抽取系統(tǒng)的實現(xiàn)研究[D];合肥工業(yè)大學;2010年
2 薛惠忠;WEB信息的抽取與集成[D];東南大學;2004年
3 史軍強;WEB信息集成技術(shù)研究[D];電子科技大學;2005年
4 賀智平;Web信息自動抽取技術(shù)研究[D];西安電子科技大學;2006年
5 江佳;信息集成中Web信息抽取技術(shù)的研究[D];西安電子科技大學;2007年
，本文編號：2245262
本文鏈接：http://sikaile.net/kejilunwen/sousuoyinqinglunwen/2245262.html
最近更新
教材專著
