【摘要】：互聯(lián)網(wǎng)技術(shù)進(jìn)步和大數(shù)據(jù)時(shí)代的到來(lái),網(wǎng)站已經(jīng)成為政府機(jī)關(guān)、企事業(yè)單位、文化傳媒、科研院校以及金融證券機(jī)構(gòu)等信息發(fā)布和綜合應(yīng)用的平臺(tái)。網(wǎng)站的使用量逐年上升,網(wǎng)頁(yè)內(nèi)容龐大繁雜,要保障網(wǎng)站信息安全、權(quán)威和準(zhǔn)確,為大眾提供正確的信息和服務(wù)是網(wǎng)站擁有者的職責(zé)所在。然而,網(wǎng)站面臨的安全威脅日趨嚴(yán)峻,非法入侵和篡改網(wǎng)站的行為層出不窮,網(wǎng)站的實(shí)時(shí)監(jiān)測(cè)及防篡改技術(shù)成為當(dāng)前信息安全領(lǐng)域中一個(gè)熱點(diǎn)的研究課題。設(shè)計(jì)研發(fā)監(jiān)測(cè)網(wǎng)站異常變化的系統(tǒng)對(duì)網(wǎng)站安全問題意義重大。對(duì)此,本文提出了一整套網(wǎng)站異常變化監(jiān)測(cè)系統(tǒng)的方案來(lái)解決這一問題。首先本文研究了網(wǎng)頁(yè)異常變化的特征,查詢了多種網(wǎng)站內(nèi)容安全保障系統(tǒng)軟件的原理和技術(shù),通過(guò)綜合的優(yōu)缺點(diǎn)比對(duì)和研究,最終選定了基于Hadoop平臺(tái)的網(wǎng)站異常變化監(jiān)測(cè)系統(tǒng)。本系統(tǒng)預(yù)期實(shí)現(xiàn)的主要功能包括網(wǎng)站文件數(shù)據(jù)的獲取、異常變化檢測(cè)以及監(jiān)測(cè)報(bào)警。其中包括爬取大量完整的網(wǎng)站文件數(shù)據(jù)、對(duì)文件數(shù)據(jù)進(jìn)行HDFS存儲(chǔ),并進(jìn)行初步過(guò)濾,再檢測(cè)出網(wǎng)站的具體變化內(nèi)容,以及變化的合法性判斷、異常數(shù)據(jù)的管理等,利用Hadoop平臺(tái)提供的文件管理系統(tǒng)HDFS和MapReduce分布式計(jì)算模型,對(duì)大量的網(wǎng)站文件數(shù)據(jù)進(jìn)行處理。系統(tǒng)對(duì)爬取到的大量網(wǎng)站文件數(shù)據(jù)進(jìn)行HDFS存儲(chǔ),并通過(guò)索引存儲(chǔ)方式加快數(shù)據(jù)搜索。系統(tǒng)進(jìn)行異常變化檢測(cè)時(shí)使用MD5信息摘要算法和改進(jìn)的基于圖論的文本比較算法,結(jié)合MapReduce計(jì)算模型實(shí)現(xiàn)快速準(zhǔn)確的異常變化檢測(cè),對(duì)非法鏈接的判斷采用URL地址轉(zhuǎn)換成IP地址分析,對(duì)非法詞匯的判斷采用匹配過(guò)濾、中文分詞技術(shù)與數(shù)據(jù)挖掘中樸素貝葉斯分類算法相結(jié)合,分類過(guò)濾出異常變化信息。通過(guò)系統(tǒng)設(shè)計(jì)、實(shí)現(xiàn)和測(cè)試,系統(tǒng)在功能和性能方面基本滿足監(jiān)測(cè)網(wǎng)站異常變化的需求,系統(tǒng)在使用中也表現(xiàn)出穩(wěn)定、高效、無(wú)差錯(cuò)地運(yùn)行。
[Abstract]:With the development of Internet technology and the arrival of big data era, website has become a platform for government agencies, enterprises and institutions, cultural media, scientific research institutions, financial and securities institutions and other information dissemination and comprehensive application platform. The usage of websites is increasing year by year, and the content of web pages is huge and complicated. It is the responsibility of website owners to ensure the security, authority and accuracy of website information and to provide correct information and services for the public. However, the security threats to websites are becoming more and more serious, and the behavior of illegal intrusion and tampering is emerging in endlessly. The real-time monitoring and tamper-proof technology of websites has become a hot research topic in the field of information security. The design and development of a system to monitor the abnormal changes of websites is of great significance to the security of websites. In this paper, a set of abnormal change monitoring system is proposed to solve this problem. Firstly, this paper studies the characteristics of the abnormal changes of web pages, inquires the principles and techniques of the software of various kinds of website content security system, and finally selects the monitoring system of the abnormal changes of the website based on Hadoop platform through the comprehensive comparison and research of the advantages and disadvantages. The main functions expected to be realized in this system include the acquisition of website file data, abnormal change detection and monitoring and alarm. These include crawling a large number of complete website file data, storing the file data by HDFS, and carrying out preliminary filtering, and then detecting the specific changing content of the website, as well as the legitimacy judgment of the change, the management of abnormal data, and so on. Using the distributed computing model of HDFS and MapReduce, a file management system provided by Hadoop platform is used to deal with a large number of web site file data. The system carries on the HDFS storage to a large number of website file data which crawls, and speeds up the data search through the index storage way. The MD5 information digest algorithm and the improved text comparison algorithm based on graph theory are used to detect the abnormal change in the system, and the MapReduce computing model is used to realize the fast and accurate anomaly change detection. To judge the illegal link, URL address is translated into IP address, matching filter is used to judge the illegal word, Chinese word segmentation technology is combined with naive Bayes classification algorithm in data mining, and abnormal information is filtered out. Through the system design, implementation and testing, the system basically meets the requirements of monitoring the abnormal changes of the website in function and performance. The system also shows stable, efficient and error-free operation in use.
【學(xué)位授予單位】：遼寧大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP393.092

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 董春濤;李文婷;沈晴霓;吳中海;;Hadoop YARN大數(shù)據(jù)計(jì)算框架及其資源調(diào)度機(jī)制研究[J];信息通信技術(shù);2015年01期

2 黃愛明;;基于軟件測(cè)試的策略與測(cè)試方法應(yīng)用分析[J];電腦知識(shí)與技術(shù);2015年02期

3 趙明芳;王學(xué)明;劉銳;;文本比較算法分析[J];電子世界;2014年04期

4 戴艷芳;;軟件可靠性與測(cè)試方法探析[J];軟件導(dǎo)刊;2012年11期

5 郝樹魁;;Hadoop HDFS和MapReduce架構(gòu)淺析[J];郵電設(shè)計(jì)技術(shù);2012年07期

6 薛輝;鄧軍;葉柏龍;;一種分布式網(wǎng)站安全防護(hù)系統(tǒng)[J];計(jì)算機(jī)系統(tǒng)應(yīng)用;2012年03期

7 陳琳;王箭;;三種中文文本自動(dòng)分類算法的比較和研究[J];計(jì)算機(jī)與現(xiàn)代化;2012年02期

8 郝大志;;網(wǎng)絡(luò)數(shù)據(jù)庫(kù)的安全管理[J];科技創(chuàng)新與應(yīng)用;2012年02期

9 侯建;帥仁俊;侯文;;基于云計(jì)算的海量數(shù)據(jù)存儲(chǔ)模型[J];通信技術(shù);2011年05期

10 李彬;;垃圾短信過(guò)濾器的研究與實(shí)現(xiàn)[J];科技傳播;2011年01期

相關(guān)碩士學(xué)位論文 前10條

1 吳俊;基于Hadoop的MapReduce作業(yè)調(diào)度系統(tǒng)的研究與應(yīng)用[D];南京郵電大學(xué);2016年

2 靳佩瑤;基于內(nèi)容的網(wǎng)頁(yè)文本信息過(guò)濾技術(shù)研究[D];西南石油大學(xué);2015年

3 黃翼彪;開源中文分詞器的比較研究[D];鄭州大學(xué);2013年

4 靳瑞敏;網(wǎng)頁(yè)關(guān)鍵字過(guò)濾研究及改進(jìn)[D];內(nèi)蒙古大學(xué);2012年

5 童明;基于HDFS的分布式存儲(chǔ)研究與應(yīng)用[D];華中科技大學(xué);2012年

6 何超;數(shù)據(jù)管理和數(shù)據(jù)挖掘技術(shù)的研究和應(yīng)用[D];北京郵電大學(xué);2012年

7 馬松華;門戶網(wǎng)站W(wǎng)eb頁(yè)面防篡改技術(shù)的研究與實(shí)現(xiàn)[D];東華大學(xué);2012年

8 徐文強(qiáng);基于HDFS的云存儲(chǔ)系統(tǒng)研究[D];上海交通大學(xué);2011年

9 孫志堅(jiān);政務(wù)網(wǎng)隔離與監(jiān)控技術(shù)研究與應(yīng)用[D];中國(guó)海洋大學(xué);2010年

10 齊曉彤;一種主動(dòng)的網(wǎng)頁(yè)防篡改機(jī)制的研究與實(shí)現(xiàn)[D];北京交通大學(xué);2010年

，

本文編號(hào)：2155156