本文選題：大數(shù)據(jù) + 計量裝置��； 參考：《華北電力大學(xué)》2017年碩士論文

【摘要】：隨著智能電網(wǎng)基礎(chǔ)設(shè)施的不斷完善,傳感器技術(shù)、通信技術(shù)和計算機技術(shù)等在智能電網(wǎng)中得到了越來越廣泛的應(yīng)用,國家電網(wǎng)的測量、采集、監(jiān)控等系統(tǒng)中產(chǎn)生了以指數(shù)級的速度增長的數(shù)據(jù)。如何利用可擴展的方式存儲和在短時間內(nèi)對這些數(shù)據(jù)進行潛在價值的分析挖掘及結(jié)果展示等成為了一個非常重要的研究課題。目前,大數(shù)據(jù)處理技術(shù)有離線處理、內(nèi)存計算等多種模式。以Hadoop框架為代表的大數(shù)據(jù)離線處理技術(shù),可以實現(xiàn)數(shù)據(jù)分布式存儲和計算,但是在計算速度上與內(nèi)存計算具有一定的差距。電能計量裝置異常信息的多維分析和故障診斷等存在一定的時效性限制,因此如何快速的對計量裝置大數(shù)據(jù)進行多維分析和診斷,成為了研究學(xué)者和專家等關(guān)注的熱點。本文首先介紹了電能計量裝置相關(guān)數(shù)據(jù)的來源,說明了智能電網(wǎng)中以智能電表為代表的電能計量裝置設(shè)備數(shù)量極其龐大,用電信息采集系統(tǒng)中收集到的相關(guān)信息也越來越多,逐漸呈現(xiàn)出了數(shù)據(jù)量龐大、數(shù)據(jù)種類多、增長速度快等大數(shù)據(jù)特點,由于計量裝置數(shù)據(jù)的特殊性,分析處理這些數(shù)據(jù)的時效性要求也比較高。針對這些特點和需求說明了并行內(nèi)存計算相對于以Hadoop為代表的離線分析處理方式的優(yōu)勢。之后對計量裝置出現(xiàn)的異常種類進行了裝置異常分析方法的建模,并制定了計量裝置異常信息多維分析和診斷的整體方案。在計量裝置異常分析的建模中,舉例介紹了幾個典型的異常情況的診斷流程和診斷方法。然后搭建大數(shù)據(jù)平臺,利用Spark SQL和HQL對計量裝置數(shù)據(jù)進行異常特征值計算,并對計算結(jié)果進行多維分析。然后詳細(xì)介紹了對樸素貝葉斯算法的并行化過程,從而利用集群優(yōu)勢對電能計量裝置進行故障診斷。最后測試了集群異常特征值計算的性能,驗證了異常特征值計算的結(jié)果,列舉了若干多維分析示例。之后進行了Spark SQL和HQL在處理數(shù)據(jù)時的效率和資源占用情況對比,同時驗證了利用并行化的樸素貝葉斯算法進行計量裝置異常診斷的可行性,分析了內(nèi)存計算相對于單機和離線批處理的效率優(yōu)勢。最后實驗驗證了集群具有很好的加速比。
[Abstract]:With the continuous improvement of smart grid infrastructure, sensor technology, communication technology and computer technology have been more and more widely used in smart grid.Monitoring and other systems produce exponential growth of data.How to store and analyze the potential value of these data in a short time by using extensible methods has become a very important research topic.At present, big data processing technology has off-line processing, memory computing and other modes.Big data's off-line processing technology, represented by Hadoop framework, can realize distributed data storage and computing, but it has a certain gap with memory computing in computing speed.The multidimensional analysis and fault diagnosis of the abnormal information of the electric energy metering device are limited in time, so how to analyze and diagnose the measuring device big data quickly has become the focus of attention of the researchers and experts.This paper first introduces the source of the related data of the electric energy metering device, and explains that the number of the electric energy metering devices represented by the smart meter in the smart grid is extremely large, and the relevant information collected in the electric information collection system is also more and more.Big data has the characteristics of huge amount of data, many kinds of data and fast growth rate. Because of the particularity of the data of metering device, the requirement of timeliness in analyzing and processing these data is also relatively high.In view of these characteristics and requirements, the advantages of parallel memory computing compared with off-line analysis and processing mode represented by Hadoop are explained.After that, the abnormal types of metering devices are modeled, and the whole scheme of multidimensional analysis and diagnosis of abnormal information of metering devices is established.In the modeling of abnormal analysis of metrology device, several typical diagnostic procedures and methods of abnormal situation are introduced with examples.Then big data platform was built, and the outliers were calculated by Spark SQL and HQL, and the results were analyzed.Then, the parallel process of naive Bayes algorithm is introduced in detail, and the fault diagnosis of electric energy metering device is made use of the advantage of cluster.Finally, the performance of the cluster outlier eigenvalue calculation is tested, the results of the outlier eigenvalue calculation are verified, and several multi-dimensional analysis examples are given.Then, the efficiency and resource occupation of Spark SQL and HQL in data processing are compared, and the feasibility of using parallel naive Bayes algorithm to diagnose abnormal metering device is verified.The efficiency advantages of memory computing compared with single computer and offline batch processing are analyzed.Finally, the experimental results show that the cluster has a good speedup.
【學(xué)位授予單位】：華北電力大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP311.13;TM933.4

【參考文獻】

相關(guān)期刊論文 前10條

1 馮興杰;賀陽;;改進的Hadoop作業(yè)調(diào)度算法[J];計算機工程與應(yīng)用;2017年12期

2 梁娟;趙開新;陳偉;;自適應(yīng)神經(jīng)模糊推理結(jié)合PID控制的并聯(lián)機器人控制方法[J];計算機應(yīng)用研究;2016年12期

3 吳黎兵;邱鑫;葉璐瑤;王曉棟;聶雷;;基于Hadoop的SQL查詢引擎性能研究[J];華中師范大學(xué)學(xué)報(自然科學(xué)版);2016年02期

4 鄭海雁;金農(nóng);季聰;熊政;李昆明;;電力用戶用電數(shù)據(jù)分析技術(shù)及典型場景應(yīng)用[J];電網(wǎng)技術(shù);2015年11期

5 崔有文;周金海;;基于KETTLE的數(shù)據(jù)集成研究[J];計算機技術(shù)與發(fā)展;2015年04期

6 王德文;孫志偉;;電力用戶側(cè)大數(shù)據(jù)分析與并行負(fù)荷預(yù)測[J];中國電機工程學(xué)報;2015年03期

7 吳凱峰;劉萬濤;李彥虎;蘇伊鵬;肖政;裴旭斌;虎嵩林;;基于云計算的電力大數(shù)據(jù)分析技術(shù)與應(yīng)用[J];中國電力;2015年02期

8 郭思亮;宋廷山;刁艷華;;基于云計算的大數(shù)據(jù)分析相關(guān)問題探討[J];齊魯師范學(xué)院學(xué)報;2015年01期

9 彭小圣;鄧迪元;程時杰;文勁宇;李朝暉;牛林;;面向智能電網(wǎng)應(yīng)用的電力大數(shù)據(jù)關(guān)鍵技術(shù)[J];中國電機工程學(xué)報;2015年03期

10 余飛;;電信運營商大數(shù)據(jù)應(yīng)用典型案例分析[J];信息通信技術(shù);2014年06期

相關(guān)碩士學(xué)位論文 前6條

1 蔣佳軍;基于Hadoop平臺的電力大客戶用電時序數(shù)據(jù)分類研究[D];華南理工大學(xué);2015年

2 婁乾;用電計量裝置在線監(jiān)測與診斷系統(tǒng)的研究與開發(fā)[D];華北電力大學(xué);2015年

3 王曉偉;面向大數(shù)據(jù)的ETL設(shè)計與實現(xiàn)[D];華中科技大學(xué);2013年

4 喻紹U，

本文編號：1732914