本文選題：大數(shù)據(jù)分布式 + 網(wǎng)絡(luò)異常流量�。� 參考：《天津理工大學(xué)》2017年碩士論文

【摘要】：近年來,互聯(lián)網(wǎng)帶給人們豐富的共享信息資源,方便了人們的工作與生活,人們越來越離不開網(wǎng)絡(luò),網(wǎng)絡(luò)在現(xiàn)代人們的生活中扮演者越來越重要的角色。另一方面,很多人針對(duì)網(wǎng)絡(luò)進(jìn)行惡意攻擊,從中獲取利益。對(duì)于大部分普通網(wǎng)民來說,網(wǎng)絡(luò)環(huán)境日益復(fù)雜,人們不僅需要加強(qiáng)自身保護(hù)意識(shí),更加需要網(wǎng)絡(luò)安全人員對(duì)網(wǎng)絡(luò)流量進(jìn)行維護(hù)與監(jiān)管,檢測異常網(wǎng)絡(luò)流量,從而保證網(wǎng)絡(luò)使用者的上網(wǎng)安全。因此,網(wǎng)絡(luò)異常流量檢測技術(shù)具有重大意義,也是文本的主要研究課題。本文針對(duì)網(wǎng)絡(luò)中數(shù)據(jù)源多維化的特點(diǎn),將數(shù)據(jù)源多維數(shù)據(jù)的信息熵投影到不同的分類支撐向量,由于機(jī)器學(xué)習(xí)容易產(chǎn)生過度訓(xùn)練或訓(xùn)練不足這些問題,本文對(duì)比EWMA、Entropy、K-means、GMM、SVDD等異常流量檢測方法,選擇支持多維數(shù)據(jù)統(tǒng)計(jì)的非監(jiān)督式SVDD分類方法,訓(xùn)練可采用的數(shù)據(jù)集。針對(duì)訓(xùn)練分類器成本消耗大與分類檢測效果佳的沖突問題,本文選擇一種異常實(shí)時(shí)響應(yīng)訓(xùn)練的學(xué)習(xí)模式,只在有異常檢測點(diǎn)加入時(shí),才進(jìn)行重新訓(xùn)練。不僅如此,對(duì)于訓(xùn)練數(shù)據(jù)集中異常點(diǎn)的選取,采用貝葉斯網(wǎng)絡(luò)模型推理預(yù)測下一節(jié)點(diǎn)的異常概率,隨即判斷是否將異常點(diǎn)加入到訓(xùn)練集中繼續(xù)訓(xùn)練,以此整體提高異常流量檢測的效率與精度。本文主要針對(duì)的問題以及研究的創(chuàng)新點(diǎn)如下:(1)針對(duì)研究過程中,異常數(shù)據(jù)集來源困難、數(shù)據(jù)分析過程復(fù)雜的客觀條件,本文選擇大數(shù)據(jù)分布式平臺(tái)環(huán)境進(jìn)行數(shù)據(jù)的處理分析。為分析分類器異常檢測能力的表現(xiàn)效果,本文對(duì)比EWMA、Entropy、K-means、GMM、SVDD等主流的異常檢測方法,最終實(shí)驗(yàn)選定在多維信息熵構(gòu)建支撐向量基礎(chǔ)上,由SVDD方法對(duì)異常流量進(jìn)行檢測,能夠有優(yōu)于其他幾種方法的檢測效果。(2)針對(duì)分類器數(shù)據(jù)量、訓(xùn)練地與實(shí)際應(yīng)用地點(diǎn)的不同,原始的訓(xùn)練集檢測效果不能很好地檢測新數(shù)據(jù),且當(dāng)前重新訓(xùn)練所需要的資源消耗量大等等這些問題,本文采用異常實(shí)時(shí)響應(yīng)訓(xùn)練方法。只有在加入異常點(diǎn)時(shí),才進(jìn)行訓(xùn)練集的重訓(xùn)練,將檢測窗口平行移動(dòng)至異常點(diǎn),剔除最開始的檢測點(diǎn),增加當(dāng)前適用的檢測點(diǎn)。這種方法既能提高訓(xùn)練集的適應(yīng)性與準(zhǔn)確率,還能將物質(zhì)消耗,資源消耗降低。(3)針對(duì)檢測統(tǒng)計(jì)量時(shí)間相關(guān)性、異常樣本對(duì)精度影響的問題,本文采用貝葉斯網(wǎng)絡(luò)模型推理預(yù)測該異常節(jié)點(diǎn)可能會(huì)出現(xiàn)異常的概率,對(duì)檢測精度進(jìn)行優(yōu)化,重新組織加入訓(xùn)練集的異常點(diǎn)。針對(duì)異常檢測模型檢測出的異常,提出應(yīng)急響應(yīng)方法,不僅能夠檢測異常,更加能夠處理異常。
[Abstract]:In recent years, the Internet has brought people a rich share of information resources, convenient for people's work and life, people are becoming more and more inseparable from the network, the network plays a more and more important role in the life of modern people. On the other hand, a lot of people have malicious attacks on the network to gain benefits. For most ordinary netizens, The network environment is increasingly complex, people not only need to strengthen their own awareness of protection, more need network security personnel to maintain and supervise network traffic, detect abnormal network traffic, so as to ensure the network users' Internet security. Therefore, the network anomaly traffic detection technology is of great significance and is also the main research topic of the text. In view of the multi-dimensional characteristics of data sources in the network, the information entropy of data source multidimensional data is projected to different classification support vectors. Because machine learning is easy to produce excessive training or lack of training, this paper compares the abnormal flow detection methods such as EWMA, Entropy, K-means, GMM, SVDD and so on, and chooses the non supervision of multi-dimensional data statistics. SVDD classification method is used to train data set which can be used. Aiming at the conflict between the high cost of training classifier and the better effect of classification detection, this paper selects a learning model of abnormal real-time response training. It is only retrained when the exception detection point is added, not only for the selection of abnormal points in the training data set, but also for the selection of abnormal points in the training data set. The Bayesian network model is used to predict the abnormal probability of the next node, and then to judge whether the anomaly points are added to the training set to continue training to improve the efficiency and accuracy of the anomaly traffic detection. The main problems and the innovation points in this paper are as follows: (1) in the process of research, the number of abnormal data sets is difficult and the number of data is difficult. According to the analysis of the complex objective conditions, this paper selects the large data distributed platform environment for data processing and analysis. In order to analyze the performance effect of the classifier anomaly detection ability, this paper compares the EWMA, Entropy, K-means, GMM, SVDD and other mainstream anomaly detection methods. The final test is selected on the basis of the multidimensional information entropy construction support vector, and the S is based on the support vector. The VDD method is better than the other several methods to detect the abnormal traffic. (2) in view of the classifier's data quantity, the training ground is different from the actual application location, the original training set detection effect can not detect the new data well, and the current retraining needs a large amount of resource consumption and so on. The training method of abnormal real-time response. Only when the exception point is added, the training set is carried out, the detection window is moved parallel to the exception point, and the first detection point is eliminated and the current detection point is increased. This method can not only improve the adaptability and accuracy of the training set, but also reduce the consumption of material and the consumption of resources. (3) (3) In this paper, we use Bayesian network model to predict the abnormal probability of the anomaly node, optimize the detection precision and reorganize the exception points of the training set. The emergency response method is proposed for the anomaly detected by the anomaly detection model. Only to be able to detect exceptions and to be able to handle exceptions more.
【學(xué)位授予單位】：天津理工大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP393.06

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 郭紅玲;程顯毅;;多分類器選擇集成方法[J];計(jì)算機(jī)工程與應(yīng)用;2009年13期

2 呂岳,施鵬飛,趙宇明;多分類器組合的投票表決規(guī)則[J];上海交通大學(xué)學(xué)報(bào);2000年05期

3 韓宏;楊靜宇;;多分類器組合及其應(yīng)用[J];計(jì)算機(jī)科學(xué);2000年01期

4 陳剛,戚飛虎;多分類器結(jié)合的人臉識(shí)別[J];上海交通大學(xué)學(xué)報(bào);2001年02期

5 韓宏,楊靜宇,婁震;基于層次的分類器組合[J];南京理工大學(xué)學(xué)報(bào)(自然科學(xué)版);2002年01期

6 趙誼虹,程國華,史習(xí)智;多分類器融合中一種新的加權(quán)算法[J];上海交通大學(xué)學(xué)報(bào);2002年06期

7 王正群,葉暉,孫興華,楊靜宇;模糊多分類器組合[J];小型微型計(jì)算機(jī)系統(tǒng);2003年01期

8 楊利英,覃征,王向華;多分類器融合實(shí)現(xiàn)機(jī)型識(shí)別[J];計(jì)算機(jī)工程與應(yīng)用;2004年15期

9 楊利英,覃征,王衛(wèi)紅;多分類器融合系統(tǒng)設(shè)計(jì)與應(yīng)用[J];計(jì)算機(jī)工程;2005年05期

10 陳湘;;1-范數(shù)軟間隔分類器的風(fēng)險(xiǎn)[J];湖北大學(xué)學(xué)報(bào)(自然科學(xué)版);2006年02期

相關(guān)會(huì)議論文 前10條

1 王占一;徐蔚然;劉東鑫;郭軍;;一種基于兩級(jí)分類器的垃圾短信過濾方法[A];第五屆全國信息檢索學(xué)術(shù)會(huì)議論文集[C];2009年

2 翟靜;李海宏;唐常杰;陳敏敏;李智;;可驗(yàn)證對(duì)象集分類器的再訓(xùn)練演進(jìn)[A];第十九屆全國數(shù)據(jù)庫學(xué)術(shù)會(huì)議論文集（研究報(bào)告篇）[C];2002年

3 陳繼航;劉家鋒;趙巍;唐降龍;;聯(lián)機(jī)手寫識(shí)別筆段特征分類器的學(xué)習(xí)方法[A];黑龍江省計(jì)算機(jī)學(xué)會(huì)2009年學(xué)術(shù)交流年會(huì)論文集[C];2010年

4 穆明生;;基于特征集的多種分類器模型的在線筆跡認(rèn)證[A];第十屆全國信號(hào)處理學(xué)術(shù)年會(huì)（CCSP-2001）論文集[C];2001年

5 彭濤;左萬利;赫楓齡;;基于鏈接上下文的分類器主題爬行技術(shù)(英文)[A];第二十三屆中國數(shù)據(jù)庫學(xué)術(shù)會(huì)議論文集（技術(shù)報(bào)告篇）[C];2006年

6 王嵐;陳珂;遲惠生;;基于多特征組合多分類器的方法用于“與文本無關(guān)”的說話人辨認(rèn)[A];第四屆全國人機(jī)語音通訊學(xué)術(shù)會(huì)議論文集[C];1996年

7 謝秋玲;;應(yīng)用于心電圖分類的KNN-SVM分類器研究[A];2006中國控制與決策學(xué)術(shù)年會(huì)論文集[C];2006年

8 胡瓊;汪榮貴;胡韋偉;孫見青;;基于級(jí)聯(lián)分類器的快速人臉檢測方法[A];計(jì)算機(jī)技術(shù)與應(yīng)用進(jìn)展·2007——全國第18屆計(jì)算機(jī)技術(shù)與應(yīng)用（CACIS）學(xué)術(shù)會(huì)議論文集[C];2007年

9 李蘭春;王雙成;杜瑞杰;;認(rèn)知結(jié)構(gòu)評(píng)估的動(dòng)態(tài)貝葉斯網(wǎng)絡(luò)分類器方法[A];2011年中國智能自動(dòng)化學(xué)術(shù)會(huì)議論文集（第一分冊）[C];2011年

10 邵小健;段華;賀國平;;一種改進(jìn)的最少核分類器[A];中國運(yùn)籌學(xué)會(huì)第七屆學(xué)術(shù)交流會(huì)論文集（上卷）[C];2004年

相關(guān)重要報(bào)紙文章 前1條

1 黃明;精子分類器決定生男生女[N];廣東科技報(bào);2000年

相關(guān)博士學(xué)位論文 前10條

1 張非;對(duì)抗逃避攻擊的防守策略研究[D];華南理工大學(xué);2015年

2 張文博;多類別智能分類器方法研究[D];西安電子科技大學(xué);2014年

3 許勁松;智能交通中目標(biāo)檢測與分類關(guān)鍵技術(shù)研究[D];南京理工大學(xué);2014年

4 趙作林;基于圖像分析的北京地區(qū)楊樹種類識(shí)別研究[D];北京林業(yè)大學(xué);2015年

5 任亞峰;基于標(biāo)注和未標(biāo)注數(shù)椐的虛假評(píng)論識(shí)別研究[D];武漢大學(xué);2015年

6 曹鵬;不均衡數(shù)據(jù)分類方法的研究[D];東北大學(xué);2014年

7 劉明;分類器組合技術(shù)研究及其在人機(jī)交互系統(tǒng)中的應(yīng)用[D];北京交通大學(xué);2008年

8 嚴(yán)志永;在劃分?jǐn)?shù)據(jù)空間的視角下基于決策邊界的分類器研究[D];浙江大學(xué);2011年

9 王U，

本文編號(hào)：2027138