發(fā)布時(shí)間：2018-07-25 08:10

【摘要】：在大范圍部署的無線傳感器網(wǎng)絡(luò)(WSN)中,節(jié)點(diǎn)產(chǎn)生大量實(shí)時(shí)的環(huán)境特征數(shù)據(jù)流,而如何從這些大規(guī)模實(shí)時(shí)數(shù)據(jù)流中高效地發(fā)現(xiàn)異常數(shù)據(jù)正是流式計(jì)算的重要研究內(nèi)容。對(duì)此,本文提出了在Spark Stream上構(gòu)建WSN流式計(jì)算系統(tǒng)以檢測(cè)異常環(huán)境特征數(shù)據(jù);為優(yōu)化數(shù)據(jù)流的傳輸和計(jì)算效率,本文引入了壓縮網(wǎng)絡(luò)編碼技術(shù),旨在改進(jìn)大數(shù)據(jù)環(huán)境下的系統(tǒng)數(shù)據(jù)處理性能。本文主要研究內(nèi)容如下:首先,設(shè)計(jì)實(shí)現(xiàn)了 WSN的末梢數(shù)據(jù)終端以感知測(cè)量環(huán)境特征數(shù)據(jù),包括各類節(jié)點(diǎn)的物理結(jié)構(gòu)和網(wǎng)絡(luò)拓?fù)?還分析了節(jié)點(diǎn)間的基礎(chǔ)數(shù)據(jù)傳輸模型和協(xié)議。WSN末梢數(shù)據(jù)終端網(wǎng)絡(luò)規(guī)劃為環(huán)帶結(jié)構(gòu),各環(huán)帶內(nèi)按簇匯聚數(shù)據(jù)并向上傳輸,形成廣義蝶形網(wǎng)絡(luò)以獲得壓縮網(wǎng)絡(luò)編碼增益。其次,構(gòu)建了流式計(jì)算平臺(tái)來快速發(fā)現(xiàn)異常數(shù)據(jù),該平臺(tái)通過數(shù)據(jù)云網(wǎng)關(guān)接收WSN末梢數(shù)據(jù)終端匯聚的原始環(huán)境特征數(shù)據(jù)流,將同步后的數(shù)據(jù)記錄推送到Spark Stream上的流式k-means程序中實(shí)時(shí)聚類,以快速發(fā)現(xiàn)大批數(shù)據(jù)流中的異常類簇。然后,改進(jìn)了 k-means算法的微批內(nèi)聚類更新方式。提出在Spark上實(shí)現(xiàn)k-means安全區(qū)間更新優(yōu)化算法,削減單個(gè)微批數(shù)據(jù)流中的流式計(jì)算時(shí)間,使系統(tǒng)及時(shí)響應(yīng)數(shù)據(jù)流累積后的聚類模型更新。最后,在系統(tǒng)的傳輸和處理階段分別引入壓縮網(wǎng)絡(luò)編碼和譯碼重構(gòu)技術(shù)。傳輸階段中,壓縮鏈路傳輸流量,以提高系統(tǒng)傳輸效率;處理階段中,利用Spark框架實(shí)現(xiàn)解碼重構(gòu)計(jì)算,充分利用大數(shù)據(jù)框架的計(jì)算性能,減少WSN節(jié)點(diǎn)性能消耗。本文形成了一個(gè)在大數(shù)據(jù)環(huán)境下進(jìn)行WSN流式計(jì)算的異常數(shù)據(jù)實(shí)時(shí)發(fā)現(xiàn)系統(tǒng),在系統(tǒng)的高效傳輸,快速處理,可靠保證等三方面進(jìn)行了系統(tǒng)優(yōu)化。
[Abstract]:In the large-scale deployed wireless sensor network (WSN), nodes generate a large number of real-time environmental feature data streams, and how to efficiently find abnormal data from these large-scale real-time data streams is an important research content of flow computing. In this paper, a WSN streaming computing system based on Spark Stream is proposed to detect the abnormal environment characteristic data, in order to optimize the transmission and computational efficiency of the data flow, the compressed network coding technology is introduced in this paper. The purpose of this paper is to improve the performance of system data processing in big data environment. The main contents of this paper are as follows: firstly, the end data terminal of WSN is designed and implemented to perceive and measure the environmental characteristic data, including the physical structure and network topology of all kinds of nodes; The basic data transmission model between nodes and the protocol. WSN terminal network is designed as a ring band structure. The data in each ring band is aggregated by clusters and transmitted upward to form a generalized butterfly network to obtain the coding gain of the compressed network. Secondly, a flow computing platform is constructed to quickly discover the abnormal data. The platform receives the raw data stream of the WSN terminal through the data cloud gateway. The synchronous data record is pushed to the streaming k-means program on Spark Stream to cluster in real time, so as to quickly find a large number of abnormal clusters in the data stream. Then, the update method of k-means algorithm is improved. An optimization algorithm for k-means security interval updating based on Spark is proposed to reduce the flow computing time in a single microbatch data stream and to enable the system to respond to the cluster model update after data flow accumulation in a timely manner. Finally, compression network coding and decoding reconstruction techniques are introduced in the transmission and processing stages of the system. In the transmission phase, the transmission flow of the link is compressed to improve the transmission efficiency of the system. In the processing stage, the decoding and reconfiguration computation is realized by using the Spark framework to make full use of the computational performance of the big data framework and the performance consumption of the WSN nodes is reduced. In this paper, a real-time discovery system of abnormal data based on WSN flow calculation in big data environment is developed. The system is optimized in three aspects: efficient transmission, fast processing and reliability assurance.
【學(xué)位授予單位】：南京理工大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP212.9;TN929.5

【參考文獻(xiàn)】

相關(guān)期刊論文 前9條

1 郭秀明;周國民;丘耘;胡林;樊景超;;一種適宜于農(nóng)業(yè)監(jiān)測(cè)和控制的WSN應(yīng)用框架[J];農(nóng)機(jī)化研究;2014年11期

2 胡夏蕓;楊余旺;曹宏鑫;王巖;殷俊;;基于作物生長模型的WSN溫室環(huán)境智能監(jiān)控系統(tǒng)[J];計(jì)算機(jī)應(yīng)用與軟件;2014年07期

3 潘淵洋;李光輝;徐勇軍;;基于DBSCAN的環(huán)境傳感器網(wǎng)絡(luò)異常數(shù)據(jù)檢測(cè)方法[J];計(jì)算機(jī)應(yīng)用與軟件;2012年11期

4 張明;朱俊平;蔡騁;;WSN中基于壓縮感知的數(shù)據(jù)收集方案[J];計(jì)算機(jī)工程;2012年20期

5 任倩倩;李建中;程思瑤;;無線傳感器網(wǎng)絡(luò)中可容錯(cuò)的事件監(jiān)測(cè)算法[J];計(jì)算機(jī)學(xué)報(bào);2012年03期

6 吳夙慧;成穎;鄭彥寧;潘云濤;;K-means算法研究綜述[J];現(xiàn)代圖書情報(bào)技術(shù);2011年05期

7 胡海峰;楊震;;無線傳感器網(wǎng)絡(luò)中基于空間相關(guān)性的分布式壓縮感知[J];南京郵電大學(xué)學(xué)報(bào)(自然科學(xué)版);2009年06期

8 陶少國;黃佳慶;楊宗凱;喬文博;熊志強(qiáng);;網(wǎng)絡(luò)編碼研究綜述[J];小型微型計(jì)算機(jī)系統(tǒng);2008年04期

9 曹冬磊;曹建農(nóng);金蓓弘;;一種無線傳感器網(wǎng)絡(luò)中事件區(qū)域檢測(cè)的容錯(cuò)算法[J];計(jì)算機(jī)學(xué)報(bào);2007年10期

，

本文編號(hào)：2143181