【摘要】：隨著互聯(lián)網(wǎng)的高速發(fā)展,網(wǎng)絡(luò)應(yīng)用和網(wǎng)絡(luò)流量不斷增長,給人類社會生活、經(jīng)濟(jì)帶來便利的同時,也給網(wǎng)絡(luò)管理和網(wǎng)絡(luò)安全帶來巨大的挑戰(zhàn)。通聯(lián)日志是網(wǎng)絡(luò)會話產(chǎn)生的日志,能很好地從會話級別描述網(wǎng)絡(luò)。因此如何為高速、長期的通聯(lián)日志提供可靠存儲,如何基于通聯(lián)日志準(zhǔn)確地對IP社會屬性進(jìn)行標(biāo)定,對于網(wǎng)絡(luò)安全、網(wǎng)絡(luò)管理以及網(wǎng)絡(luò)規(guī)劃都起著至關(guān)重要的作用。 現(xiàn)存的面向通聯(lián)日志的存儲方案沒有很好地兼顧接收、存儲和檢索等方面,使得它們的可移植性受到限制。多個骨干網(wǎng)生成的通聯(lián)日志每秒鐘可以達(dá)到千萬級別,這使得傳統(tǒng)集中式存儲方案越來越不能滿足需求。而后的一些方案借助分布式框架提供可擴(kuò)展的存儲性能,但是這些方案的存儲引擎大多基于傳統(tǒng)關(guān)系型數(shù)據(jù)庫,存儲性能有限。本文對通聯(lián)日志存儲方案進(jìn)行深入學(xué)習(xí),以實現(xiàn)支持高速存儲、高速查詢的分布式通聯(lián)日志存儲系統(tǒng)。另外,本文對傳統(tǒng)IP社會屬性標(biāo)定進(jìn)行深入研究,發(fā)現(xiàn)傳統(tǒng)的基于端口和行為特征的IP社會屬性標(biāo)定分辨率較低。本文對通聯(lián)日志進(jìn)行詳細(xì)分析,以實現(xiàn)更加準(zhǔn)確的IP社會屬性標(biāo)定。本文的主要的研究內(nèi)容可以歸納為： (1)提出新型的高速通聯(lián)日志接收框架：DPIO (Driect Packet I/O)。盡管基于傳統(tǒng)的Socket API可以比較簡單實現(xiàn)通聯(lián)日志接收,但是其性能不高。而新型網(wǎng)絡(luò)驅(qū)動netmap,可以很好地解決這個問題,但是netmap需要單獨維護(hù)網(wǎng)卡驅(qū)動,實現(xiàn)和維護(hù)都比較困難。本文在它們的基礎(chǔ)上提出一種新型的通聯(lián)日志接收框架,實驗結(jié)果表明DPIO既能解決Socket API接收速率低的問題,同時也能避免netmap的復(fù)雜性。 (2)設(shè)計并實現(xiàn)支持高速存儲和快速檢索的分布式通聯(lián)日志存儲系統(tǒng)：DCLStore (A Distributed Connection Log Storage System Supports High-speed storage and Fast Retrieval)。DCLStore能夠為通聯(lián)日志提供高速存儲能力、高速檢索能力,并且通過存儲節(jié)點的動態(tài)增加提供可擴(kuò)展的存儲空間。實驗結(jié)果,本系統(tǒng)每秒鐘可以接收大約2000萬條通聯(lián)日志,并能很好的處理多個網(wǎng)絡(luò)節(jié)點的日志。在查詢時可以提供比相同存儲容量下的單點存儲系統(tǒng)高40倍的查詢響應(yīng)速度。 (3)提出新型的IP指標(biāo)：IP明暗度。傳統(tǒng)的基于端口和行為特征的IP社會屬性標(biāo)定雖然實現(xiàn)簡單,但是準(zhǔn)確率不高。本文首先對通聯(lián)日志進(jìn)行深入觀察,提出一種新型的IP指標(biāo)：IP明暗度。而后,本文對全網(wǎng)IP的指標(biāo)進(jìn)行基礎(chǔ)測量,測量結(jié)果表明本文對通聯(lián)日志處理的結(jié)果基本正確。最后,本文利用開源工具對全網(wǎng)IP的明暗度進(jìn)行計算,并考察其對IP屬性標(biāo)定的影響。實驗結(jié)果表明,IP明暗度對基于端口和行為特征的IP社會屬性標(biāo)定的結(jié)果都有很大的影響。
[Abstract]:With the rapid development of the Internet, network application and network traffic are increasing, which brings convenience to human life and economy, and also brings great challenges to network management and network security. The communication log is the log generated by the network session and can describe the network from the session level. Therefore, how to provide reliable storage for high-speed, long-term communication logs, and how to accurately calibrate the social attributes of IP based on communication logs play an important role in network security, network management and network planning. The existing storage schemes for communication logs do not take account of reception, storage and retrieval, so their portability is limited. Communication logs generated by multiple backbone networks can reach tens of millions of levels per second, which makes the traditional centralized storage scheme more and more unable to meet the requirements. Then some schemes provide extensible storage performance with the help of distributed framework, but most of the storage engines of these schemes are based on traditional relational databases, and the storage performance is limited. In order to realize the distributed communication log storage system which supports high speed storage and high speed query, this paper makes a deep study on the communication log storage scheme. In addition, the traditional IP social attribute calibration is deeply studied in this paper. It is found that the traditional IP social attribute calibration based on port and behavior features has lower resolution. In this paper, the connection log is analyzed in detail in order to achieve more accurate IP social attribute calibration. The main research contents of this paper can be summarized as follows: (1) A new high-speed communication log receiving framework,: DPIO (Driect Packet I / O, is proposed. Although the traditional Socket API can easily realize the communication log reception, but its performance is not high. The new network driver netmap, can solve this problem well, but the netmap needs to maintain the network card driver alone, so it is difficult to implement and maintain. In this paper, we propose a new communication log receiving framework based on them. The experimental results show that DPIO can not only solve the problem of low receiving rate of Socket API, but also avoid the complexity of netmap. (2) Design and implement the distributed connection log storage system (: DCLStore (A Distributed Connection Log Storage System Supports High-speed storage and Fast Retrieval). DCLStore), which supports high speed storage and fast retrieval, can provide high speed storage capacity and high speed retrieval ability for communication log. And provides the extensible storage space through the dynamic increase of the storage node. Experimental results show that the system can receive about 20 million communication logs per second, and can handle the logs of multiple network nodes well. The query response speed is 40 times faster than the single point storage system with the same storage capacity. (3) A new IP index, IP lightness and darkness, is proposed. The traditional IP social attribute calibration based on port and behavior is simple, but the accuracy is not high. In this paper, a new IP index, IP lightness and darkness, is proposed. Then, this paper carries on the basic measurement to the whole network IP index, the measurement result shows that the result of this paper is basically correct to the connection log processing. Finally, the open source tools are used to calculate the lightness of IP in the whole network and its influence on IP attribute calibration is investigated. The experimental results show that the IP lightness has great influence on the results of IP social attribute calibration based on port and behavioral characteristics.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2015
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前2條

1 李現(xiàn)艷;趙書俊;初元萍;;基于MySQL的數(shù)據(jù)庫服務(wù)器性能測試[J];核電子學(xué)與探測技術(shù);2011年01期

2 徐非,楊廣文,鞠大鵬;基于Peer-to-Peer的分布式存儲系統(tǒng)的設(shè)計[J];軟件學(xué)報;2004年02期

，

本文編號：2322178