【摘要】：隨著全球能源互聯(lián)網(wǎng)的建設(shè)以及智能電網(wǎng)建設(shè)的快速推進(jìn),大量的物聯(lián)網(wǎng)信息采集設(shè)備終端將會(huì)接入電網(wǎng),這些終端將會(huì)產(chǎn)生海量的采集數(shù)據(jù)——智能電網(wǎng)大數(shù)據(jù)。為應(yīng)對(duì)這些海量數(shù)據(jù)分析的需求,本文研究了智能電網(wǎng)大數(shù)據(jù)的流處理和批處理引擎的構(gòu)建,并在此基礎(chǔ)上完成了智能電網(wǎng)大數(shù)據(jù)在線分析與決策系統(tǒng)的設(shè)計(jì)。本文在國(guó)內(nèi)外的研究的基礎(chǔ)上研究了智能電網(wǎng)大數(shù)據(jù)的來源和分類,對(duì)智能電網(wǎng)大數(shù)據(jù)分析的主要需求進(jìn)行了分析。本文對(duì)大數(shù)據(jù)相關(guān)的分布式計(jì)算理論進(jìn)行了介紹,其中主要介紹了分布式計(jì)算框架MapReduce、分布式文件系統(tǒng)GFS和HDFS、分布式應(yīng)用程序協(xié)調(diào)服務(wù)Chubby和ZooKeeper、分布式資源管理框架YARN和Mesos的原理和架構(gòu),同時(shí)也介紹了Map Reduce迭代計(jì)算模型、BSP計(jì)算模型、SSP計(jì)算模型這三種分布式數(shù)據(jù)分算法基礎(chǔ)模型;隨后研究了智能電網(wǎng)大數(shù)據(jù)流處理的任務(wù)需求,對(duì)流處理的概念進(jìn)行了介紹,同時(shí)研究了智能電網(wǎng)大數(shù)據(jù)流處理系統(tǒng)的需求特征,著重研究了Strom、Spark Streaming、Samza這三種流處理引擎以及其應(yīng)用場(chǎng)景,根據(jù)其特點(diǎn)和智能電網(wǎng)大數(shù)據(jù)流處理分析的特性需求本文選擇Strom作為構(gòu)建智能大數(shù)據(jù)在線分析與決策系統(tǒng)的流處理引擎,并通過基于Storm的VFDT算法在重要電力客戶供用電安全實(shí)時(shí)分析中的應(yīng)用展示了Strom在電網(wǎng)數(shù)據(jù)實(shí)時(shí)分析上的有效性,通過機(jī)器的擴(kuò)展和模擬數(shù)據(jù)流的增加進(jìn)行壓力測(cè)試證明了Strom流處理引擎的在智能電網(wǎng)大數(shù)據(jù)分析場(chǎng)景中的可擴(kuò)展性;隨后,對(duì)智能電網(wǎng)大數(shù)據(jù)批處理的任務(wù)需求進(jìn)行了研究,提出了利用Spark建設(shè)智能電網(wǎng)大數(shù)據(jù)批處理分析引擎的方案,并通過基于Spark的隨機(jī)森林算法在海量用電負(fù)荷數(shù)據(jù)分析中的應(yīng)用驗(yàn)證了該解決方案的有效性、可擴(kuò)展性;最后,在以上研究的基礎(chǔ)上對(duì)智能電網(wǎng)大數(shù)據(jù)在線分析與決策系統(tǒng)進(jìn)行了詳細(xì)的需求分析,并設(shè)計(jì)了系統(tǒng)的整體架構(gòu)和每個(gè)模塊的功能,這一設(shè)計(jì)方案可為后續(xù)的軟件開發(fā)提供直接的參考依據(jù)。
[Abstract]:With the construction of the global energy Internet and the rapid development of the smart grid, a large number of Internet of things information acquisition equipment terminals will be connected to the grid, these terminals will produce a huge amount of data acquisition-big data smart grid. In order to meet the demand of these massive data analysis, this paper studies the construction of big data's flow processing and batch processing engine of smart grid, and on this basis completes the design of online analysis and decision making system for big data in smart grid. On the basis of domestic and international research, this paper studies the source and classification of big data in smart grid, and analyzes the main requirements of the analysis of the intelligent power grid big data. This paper introduces the distributed computing theory related to big data, including the distributed computing framework MapReduce, distributed file system GFS and HDFS, distributed application coordination service Chubby and ZooKeeper,. The principle and architecture of distributed resource management framework (YARN and Mesos) are introduced, and three basic models of distributed data division algorithm, namely, Map Reduce iterative computing model, BSP computing model and SSP computing model, are also introduced. Then, the task requirements of large data flow processing in smart grid and the concept of convection processing are introduced. At the same time, the demand characteristics of large data flow processing system in smart grid are studied, and the Strom,Spark Streaming, is emphatically studied. According to the characteristics of Samza and its application scenarios, this paper chooses Strom as the flow processing engine to construct the intelligent big data online analysis and decision system according to its characteristics and the characteristics of large data flow processing and analysis of smart grid. The application of VFDT algorithm based on Storm in real-time analysis of power supply and power security of important power customers shows the effectiveness of Strom in real-time analysis of power network data. The expansibility of Strom flow processing engine in the analysis scene of smart grid big data is proved by the expansion of the machine and the increase of simulated data flow. Then, the task requirement of big data batch processing in smart grid is studied, and the scheme of building a batch processing engine based on Spark is put forward. The application of stochastic forest algorithm based on Spark in the analysis of massive power load data proves the validity and expansibility of the solution. Finally, on the basis of the above research, a detailed requirement analysis of big data online analysis and decision-making system of smart grid is carried out, and the overall structure of the system and the function of each module are designed. This design can provide a direct reference for the subsequent software development.
【學(xué)位授予單位】：華北電力大學(xué)(北京)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP311.13;F426.61

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 張發(fā)揚(yáng);李玲娟;陳煜;;VFDT算法基于Storm平臺(tái)的實(shí)現(xiàn)方案[J];計(jì)算機(jī)技術(shù)與發(fā)展;2016年09期

2 Marie-Luce PICARD;潘旭陽;;從法國(guó)公共電力企業(yè)的視角看大數(shù)據(jù)帶來的挑戰(zhàn)和機(jī)遇[J];電網(wǎng)技術(shù);2015年11期

3 鄭海雁;金農(nóng);季聰;熊政;李昆明;;電力用戶用電數(shù)據(jù)分析技術(shù)及典型場(chǎng)景應(yīng)用[J];電網(wǎng)技術(shù);2015年11期

4 王德文;孫志偉;;電力用戶側(cè)大數(shù)據(jù)分析與并行負(fù)荷預(yù)測(cè)[J];中國(guó)電機(jī)工程學(xué)報(bào);2015年03期

5 張東霞;苗新;劉麗平;張焰;劉科研;;智能電網(wǎng)大數(shù)據(jù)技術(shù)發(fā)展研究[J];中國(guó)電機(jī)工程學(xué)報(bào);2015年01期

6 劉科研;盛萬興;張東霞;賈東梨;胡麗娟;何開元;;智能配電網(wǎng)大數(shù)據(jù)應(yīng)用需求和場(chǎng)景分析研究[J];中國(guó)電機(jī)工程學(xué)報(bào);2015年02期

7 宋亞奇;周國(guó)亮;朱永利;李莉;王劉旺;王德文;;云平臺(tái)下輸變電設(shè)備狀態(tài)監(jiān)測(cè)大數(shù)據(jù)存儲(chǔ)優(yōu)化與并行處理[J];中國(guó)電機(jī)工程學(xué)報(bào);2015年02期

8 欒文鵬;余貽鑫;王兵;;AMI數(shù)據(jù)分析方法[J];中國(guó)電機(jī)工程學(xué)報(bào);2015年01期

9 曲朝陽;陳帥;楊帆;朱莉;;基于云計(jì)算技術(shù)的電力大數(shù)據(jù)預(yù)處理屬性約簡(jiǎn)方法[J];電力系統(tǒng)自動(dòng)化;2014年08期

10 楊勁鋒;劉濤;陳啟冠;闕華坤;肖勇;;基于海量計(jì)量數(shù)據(jù)的電力客戶在線分群研究[J];華東電力;2013年08期

相關(guān)碩士學(xué)位論文 前6條

1 李勁松;一種基于Storm的分布式實(shí)時(shí)增量計(jì)算框架的研究與實(shí)現(xiàn)[D];電子科技大學(xué);2015年

2 馮懿;基于云計(jì)算的電力系統(tǒng)不良數(shù)據(jù)辨識(shí)算法研究[D];南京理工大學(xué);2013年

3 湛維明;云計(jì)算環(huán)境下的發(fā)電優(yōu)化調(diào)度并行算法研究[D];華北電力大學(xué);2013年

4 白紅偉;基于云計(jì)算的電力設(shè)備狀態(tài)監(jiān)測(cè)數(shù)據(jù)的存儲(chǔ)與查詢[D];華北電力大學(xué);2012年

5 趙黎斌;面向云存儲(chǔ)的分布式文件系統(tǒng)關(guān)鍵技術(shù)研究[D];西安電子科技大學(xué);2011年

6 劉芳;基于數(shù)據(jù)挖掘的電網(wǎng)數(shù)據(jù)智能分析的研究與實(shí)現(xiàn)[D];西北大學(xué);2008年

，

本文編號(hào)：2307327