本文選題：計(jì)算機(jī)海量數(shù)據(jù)　切入點(diǎn)：流計(jì)算　出處：《北京郵電大學(xué)》2014年碩士論文

【摘要】：隨著網(wǎng)絡(luò)信息的急劇膨脹,給網(wǎng)絡(luò)的管理也帶來(lái)了前所未有的挑戰(zhàn)。傳統(tǒng)的網(wǎng)絡(luò)管理軟件有擴(kuò)展性差,分析網(wǎng)絡(luò)數(shù)據(jù)方式固定化,用戶無(wú)法自定義數(shù)據(jù)分析方式等問(wèn)題。針對(duì)以上問(wèn)題,本文提出了一種支持海量數(shù)據(jù)實(shí)時(shí)處理的網(wǎng)絡(luò)分析系統(tǒng)和方法來(lái)彌補(bǔ)這個(gè)領(lǐng)域的不足。 針對(duì)現(xiàn)有網(wǎng)絡(luò)分析軟件存在的缺陷,本文提出一種基于“流水線”思想的網(wǎng)絡(luò)分析系統(tǒng)平臺(tái)的設(shè)計(jì)方案。同時(shí),該方案結(jié)合P2P技術(shù)和事件處理機(jī)制等進(jìn)行設(shè)計(jì)和實(shí)現(xiàn)。該平臺(tái)的工作方式類(lèi)似于“流水線”,每個(gè)數(shù)據(jù)分析流程由若干個(gè)相對(duì)獨(dú)立的數(shù)據(jù)分析節(jié)點(diǎn)來(lái)一環(huán)扣一環(huán)的逐步分析完成；其底層的各數(shù)據(jù)分析邏輯間通過(guò)統(tǒng)一的事件機(jī)制來(lái)通信協(xié)作。同時(shí),為了實(shí)現(xiàn)用戶自定義數(shù)據(jù)分析方式,系統(tǒng)增加邏輯管理模塊,用戶通過(guò)擴(kuò)展通用的模板接口,可以非常方便的完成自定義數(shù)據(jù)處理的需求。 為實(shí)現(xiàn)上述設(shè)計(jì),本文通過(guò)兩種數(shù)據(jù)節(jié)點(diǎn)完成系統(tǒng)的功能：數(shù)據(jù)適配節(jié)點(diǎn)和數(shù)據(jù)分析節(jié)點(diǎn)。數(shù)據(jù)適配節(jié)點(diǎn)主要完成數(shù)據(jù)和內(nèi)部事件之間的適配。數(shù)據(jù)分析節(jié)點(diǎn)是來(lái)完成數(shù)據(jù)分析任務(wù),我們使用數(shù)據(jù)分析節(jié)點(diǎn)的可擴(kuò)展性和可自定義處理邏輯性來(lái)完成流水線的搭建,完成數(shù)據(jù)的監(jiān)視分析等操作。本文通過(guò)類(lèi)圖和流程圖兩種方式展開(kāi)實(shí)現(xiàn)的闡述。最后通過(guò)網(wǎng)絡(luò)數(shù)據(jù)包超時(shí)的實(shí)例測(cè)試,完成功能的校驗(yàn)及性能的測(cè)試。 本文闡述的方案特色：1、用戶可以根據(jù)自己的要求,通過(guò)我們提供的接口自定義數(shù)據(jù)分析邏輯,將其注入容器,即可生效；2、本系統(tǒng)可以通過(guò)動(dòng)態(tài)擴(kuò)展節(jié)點(diǎn)來(lái)提高性能的要求。由于有流水線的作業(yè)方式和事件路由機(jī)制提供的支持,只需要積木式的增加服務(wù)器,數(shù)據(jù)適配節(jié)點(diǎn)就會(huì)動(dòng)態(tài)的將負(fù)載轉(zhuǎn)移到新增加的服務(wù)器上,從而到達(dá)動(dòng)態(tài)擴(kuò)展節(jié)點(diǎn)來(lái)提高性能的要求。
[Abstract]:With the rapid expansion of network information, it brings unprecedented challenges to network management. To solve the above problems, a network analysis system and method to support real-time processing of massive data is proposed to make up for the shortcomings in this field. In view of the defects of the existing network analysis software, this paper proposes a design scheme of the network analysis system platform based on pipeline's thought. This scheme is designed and implemented with P2P technology and event handling mechanism. The working mode of this platform is similar to that of "pipeline". Each data analysis process is completed by several independent data analysis nodes step by step. At the same time, in order to realize the user-defined data analysis mode, the system adds the logical management module, and the user extends the common template interface. You can easily complete the requirements of custom data processing. In order to implement the above design, In this paper, two kinds of data nodes are used to complete the functions of the system: the data adaptation node and the data analysis node. The data adaptation node mainly completes the adaptation between the data and the internal events. The data analysis node is to complete the data analysis task. We use the extensibility and customizable processing logic of the data analysis node to complete pipeline's construction. Through class diagram and flow chart, the implementation is described. Finally, the function verification and performance test are completed by the example test of network data packet timeout. The solution described in this article features: 1, the user can customize the data analysis logic through the interface we provide, and inject it into the container according to his own request. This system can improve the performance requirements by dynamically extending nodes. With the support of pipeline's operation mode and event routing mechanism, only building blocks are needed to increase the server. The data adaptation node will dynamically transfer the load to the newly added server, thus reaching the requirement of dynamic expansion node to improve the performance.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2014
【分類(lèi)號(hào)】：TP393.07

【參考文獻(xiàn)】

相關(guān)期刊論文 前7條

1 龔尚福;朱莉;李娜;;簡(jiǎn)單網(wǎng)絡(luò)管理協(xié)議的研究與應(yīng)用[J];電子設(shè)計(jì)工程;2009年07期

2 段文軍;;分布式網(wǎng)絡(luò)管理體系結(jié)構(gòu)的研究與分析[J];電腦與電信;2011年06期

3 黃明輝;;基于SNMP的網(wǎng)絡(luò)故障管理系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];遼寧大學(xué)學(xué)報(bào)(自然科學(xué)版);2012年03期

4 李林峰;位偉鋒;;sniffer嗅探工具在網(wǎng)絡(luò)管理工作中的應(yīng)用[J];河南科技;2013年11期

5 羅青林;徐克付;臧文羽;劉金剛;;Wireshark環(huán)境下的網(wǎng)絡(luò)協(xié)議解析與驗(yàn)證方法[J];計(jì)算機(jī)工程與設(shè)計(jì);2011年03期

6 楊妮妮;趙新慧;李文超;馮錫煒;;基于Web的SNMP網(wǎng)絡(luò)管理[J];微計(jì)算機(jī)信息;2006年06期

7 嚴(yán)有日;;網(wǎng)絡(luò)嗅探器Sniffer技術(shù)分析及研究[J];西安文理學(xué)院學(xué)報(bào)(自然科學(xué)版);2011年01期

相關(guān)博士學(xué)位論文 前2條

1 馬杰;網(wǎng)絡(luò)安全威脅態(tài)勢(shì)評(píng)估與分析方法研究[D];華中科技大學(xué);2010年

2 楊天路;網(wǎng)絡(luò)威脅檢測(cè)與防御關(guān)鍵技術(shù)研究[D];北京郵電大學(xué);2010年

，

本文編號(hào)：1688273