【摘要】：互聯(lián)網(wǎng)技術(shù)進步和大數(shù)據(jù)時代的到來,網(wǎng)站已經(jīng)成為政府機關(guān)、企事業(yè)單位、文化傳媒、科研院校以及金融證券機構(gòu)等信息發(fā)布和綜合應(yīng)用的平臺。網(wǎng)站的使用量逐年上升,網(wǎng)頁內(nèi)容龐大繁雜,要保障網(wǎng)站信息安全、權(quán)威和準確,為大眾提供正確的信息和服務(wù)是網(wǎng)站擁有者的職責所在。然而,網(wǎng)站面臨的安全威脅日趨嚴峻,非法入侵和篡改網(wǎng)站的行為層出不窮,網(wǎng)站的實時監(jiān)測及防篡改技術(shù)成為當前信息安全領(lǐng)域中一個熱點的研究課題。設(shè)計研發(fā)監(jiān)測網(wǎng)站異常變化的系統(tǒng)對網(wǎng)站安全問題意義重大。對此,本文提出了一整套網(wǎng)站異常變化監(jiān)測系統(tǒng)的方案來解決這一問題。首先本文研究了網(wǎng)頁異常變化的特征,查詢了多種網(wǎng)站內(nèi)容安全保障系統(tǒng)軟件的原理和技術(shù),通過綜合的優(yōu)缺點比對和研究,最終選定了基于Hadoop平臺的網(wǎng)站異常變化監(jiān)測系統(tǒng)。本系統(tǒng)預(yù)期實現(xiàn)的主要功能包括網(wǎng)站文件數(shù)據(jù)的獲取、異常變化檢測以及監(jiān)測報警。其中包括爬取大量完整的網(wǎng)站文件數(shù)據(jù)、對文件數(shù)據(jù)進行HDFS存儲,并進行初步過濾,再檢測出網(wǎng)站的具體變化內(nèi)容,以及變化的合法性判斷、異常數(shù)據(jù)的管理等,利用Hadoop平臺提供的文件管理系統(tǒng)HDFS和MapReduce分布式計算模型,對大量的網(wǎng)站文件數(shù)據(jù)進行處理。系統(tǒng)對爬取到的大量網(wǎng)站文件數(shù)據(jù)進行HDFS存儲,并通過索引存儲方式加快數(shù)據(jù)搜索。系統(tǒng)進行異常變化檢測時使用MD5信息摘要算法和改進的基于圖論的文本比較算法,結(jié)合MapReduce計算模型實現(xiàn)快速準確的異常變化檢測,對非法鏈接的判斷采用URL地址轉(zhuǎn)換成IP地址分析,對非法詞匯的判斷采用匹配過濾、中文分詞技術(shù)與數(shù)據(jù)挖掘中樸素貝葉斯分類算法相結(jié)合,分類過濾出異常變化信息。通過系統(tǒng)設(shè)計、實現(xiàn)和測試,系統(tǒng)在功能和性能方面基本滿足監(jiān)測網(wǎng)站異常變化的需求,系統(tǒng)在使用中也表現(xiàn)出穩(wěn)定、高效、無差錯地運行。
[Abstract]:With the development of Internet technology and the arrival of big data era, website has become a platform for government agencies, enterprises and institutions, cultural media, scientific research institutions, financial and securities institutions and other information dissemination and comprehensive application platform. The usage of websites is increasing year by year, and the content of web pages is huge and complicated. It is the responsibility of website owners to ensure the security, authority and accuracy of website information and to provide correct information and services for the public. However, the security threats to websites are becoming more and more serious, and the behavior of illegal intrusion and tampering is emerging in endlessly. The real-time monitoring and tamper-proof technology of websites has become a hot research topic in the field of information security. The design and development of a system to monitor the abnormal changes of websites is of great significance to the security of websites. In this paper, a set of abnormal change monitoring system is proposed to solve this problem. Firstly, this paper studies the characteristics of the abnormal changes of web pages, inquires the principles and techniques of the software of various kinds of website content security system, and finally selects the monitoring system of the abnormal changes of the website based on Hadoop platform through the comprehensive comparison and research of the advantages and disadvantages. The main functions expected to be realized in this system include the acquisition of website file data, abnormal change detection and monitoring and alarm. These include crawling a large number of complete website file data, storing the file data by HDFS, and carrying out preliminary filtering, and then detecting the specific changing content of the website, as well as the legitimacy judgment of the change, the management of abnormal data, and so on. Using the distributed computing model of HDFS and MapReduce, a file management system provided by Hadoop platform is used to deal with a large number of web site file data. The system carries on the HDFS storage to a large number of website file data which crawls, and speeds up the data search through the index storage way. The MD5 information digest algorithm and the improved text comparison algorithm based on graph theory are used to detect the abnormal change in the system, and the MapReduce computing model is used to realize the fast and accurate anomaly change detection. To judge the illegal link, URL address is translated into IP address, matching filter is used to judge the illegal word, Chinese word segmentation technology is combined with naive Bayes classification algorithm in data mining, and abnormal information is filtered out. Through the system design, implementation and testing, the system basically meets the requirements of monitoring the abnormal changes of the website in function and performance. The system also shows stable, efficient and error-free operation in use.
【學位授予單位】：遼寧大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP393.092

【參考文獻】

相關(guān)期刊論文 前10條

1 董春濤;李文婷;沈晴霓;吳中海;;Hadoop YARN大數(shù)據(jù)計算框架及其資源調(diào)度機制研究[J];信息通信技術(shù);2015年01期

2 黃愛明;;基于軟件測試的策略與測試方法應(yīng)用分析[J];電腦知識與技術(shù);2015年02期

3 趙明芳;王學明;劉銳;;文本比較算法分析[J];電子世界;2014年04期

4 戴艷芳;;軟件可靠性與測試方法探析[J];軟件導(dǎo)刊;2012年11期

5 郝樹魁;;Hadoop HDFS和MapReduce架構(gòu)淺析[J];郵電設(shè)計技術(shù);2012年07期

6 薛輝;鄧軍;葉柏龍;;一種分布式網(wǎng)站安全防護系統(tǒng)[J];計算機系統(tǒng)應(yīng)用;2012年03期

7 陳琳;王箭;;三種中文文本自動分類算法的比較和研究[J];計算機與現(xiàn)代化;2012年02期

8 郝大志;;網(wǎng)絡(luò)數(shù)據(jù)庫的安全管理[J];科技創(chuàng)新與應(yīng)用;2012年02期

9 侯建;帥仁俊;侯文;;基于云計算的海量數(shù)據(jù)存儲模型[J];通信技術(shù);2011年05期

10 李彬;;垃圾短信過濾器的研究與實現(xiàn)[J];科技傳播;2011年01期

相關(guān)碩士學位論文 前10條

1 吳俊;基于Hadoop的MapReduce作業(yè)調(diào)度系統(tǒng)的研究與應(yīng)用[D];南京郵電大學;2016年

2 靳佩瑤;基于內(nèi)容的網(wǎng)頁文本信息過濾技術(shù)研究[D];西南石油大學;2015年

3 黃翼彪;開源中文分詞器的比較研究[D];鄭州大學;2013年

4 靳瑞敏;網(wǎng)頁關(guān)鍵字過濾研究及改進[D];內(nèi)蒙古大學;2012年

5 童明;基于HDFS的分布式存儲研究與應(yīng)用[D];華中科技大學;2012年

6 何超;數(shù)據(jù)管理和數(shù)據(jù)挖掘技術(shù)的研究和應(yīng)用[D];北京郵電大學;2012年

7 馬松華;門戶網(wǎng)站W(wǎng)eb頁面防篡改技術(shù)的研究與實現(xiàn)[D];東華大學;2012年

8 徐文強;基于HDFS的云存儲系統(tǒng)研究[D];上海交通大學;2011年

9 孫志堅;政務(wù)網(wǎng)隔離與監(jiān)控技術(shù)研究與應(yīng)用[D];中國海洋大學;2010年

10 齊曉彤;一種主動的網(wǎng)頁防篡改機制的研究與實現(xiàn)[D];北京交通大學;2010年

，

本文編號：2155156