發(fā)布時間：2018-11-22 14:15

【摘要】：降雨量是防災(zāi)減災(zāi)的重要參量,很大程度反映災(zāi)害發(fā)生趨勢,降雨量對農(nóng)業(yè)生產(chǎn)、水土流式和工程應(yīng)用等有著重要的影響,對一個地區(qū)的降雨量進行準確預(yù)測,可以幫助農(nóng)業(yè)、水利部門提高防治旱澇災(zāi)害的能力,將危害降低到最低。隨著近幾年,我國洪澇災(zāi)害不斷頻發(fā),如何準確及時地利用氣象數(shù)據(jù)對降雨量預(yù)報也變得越來越重要了。大數(shù)據(jù)時代的來臨,也給氣象預(yù)報行業(yè)帶來了新的挑戰(zhàn)。氣象數(shù)據(jù)主要來自于地面觀測、氣象衛(wèi)星遙感、天氣雷達和數(shù)值預(yù)報產(chǎn)品。這四類數(shù)據(jù)占數(shù)據(jù)總量的90%以上,直接應(yīng)用于氣象業(yè)務(wù)、天氣預(yù)報、氣候預(yù)測以及氣象服務(wù)。流數(shù)據(jù)是一組數(shù)字編碼并連續(xù)的信號。一般情況下,數(shù)據(jù)流可被視為一個隨時間延續(xù)而無限廣泛應(yīng)用于網(wǎng)絡(luò)輿情分析、股票市場走向、衛(wèi)星定位、金融實時監(jiān)控、物聯(lián)網(wǎng)監(jiān)控以及實時氣象監(jiān)控等多個領(lǐng)域。在基于大規(guī)模氣象流數(shù)據(jù)的降雨量預(yù)測領(lǐng)域,還有很大的發(fā)展空間。對于傳統(tǒng)的降雨量預(yù)測,往往利用離線的氣象數(shù)據(jù),采用機器學(xué)習的方法進行批量訓(xùn)練,即所有的訓(xùn)練樣本一次性學(xué)習完畢后,學(xué)習過程不再繼續(xù)。但在實際應(yīng)用中,訓(xùn)練樣本空間的全部樣本并不能一次得到,而往往是隨著時間順序得到。盡管采用大規(guī)模集群能夠在一定程度上緩解大量數(shù)據(jù)帶來計算能力不足的問題,但是對于新到達的數(shù)據(jù),卻不能進行快速處理學(xué)習并及時更新學(xué)習所獲得的知識。針對氣象數(shù)據(jù)的實時計算與海量處理的問題,本文提出了一種基于Storm平臺的在線序列的極限學(xué)習機降雨量預(yù)測模型。本文的主要內(nèi)容和創(chuàng)新點如下:(1)針對氣象數(shù)據(jù)的離線批量預(yù)測方法不能及時反映降雨量變化的問題,提出了一種基于在線序列極限學(xué)習機的降雨量預(yù)測模型。針對氣象數(shù)據(jù)的大規(guī)模和實時特性,對極限學(xué)習機算法進行在線序列優(yōu)化。該模型首先初始化多個在線極限學(xué)習機模型,當不斷到達新的批次的數(shù)據(jù)時,模型能夠在已有的訓(xùn)練結(jié)果的基礎(chǔ)上繼續(xù)學(xué)習新樣本,并引入隨機梯度下降法和誤差權(quán)值調(diào)整的方式,對新的預(yù)測結(jié)果進行誤差反饋,實時更新誤差權(quán)值參數(shù),以提升模型預(yù)測準確率。(2)針對氣象數(shù)據(jù)的海量高維特性的問題,在數(shù)據(jù)預(yù)處理階段,本文采用決策屬性之間的相關(guān)系數(shù)對氣象數(shù)據(jù)分析,利用相關(guān)系數(shù)篩選預(yù)測屬性,降低了氣象數(shù)據(jù)復(fù)雜度,提高了模型訓(xùn)練效率。另外,采用Storm流式大數(shù)據(jù)處理框架結(jié)合Kafka分布式消息隊列,對大規(guī)模氣象數(shù)據(jù)進行并行訓(xùn)練。實驗結(jié)果表明,算法在Storm平臺上運行,具有優(yōu)異的并行性能和預(yù)測精度。
[Abstract]:Rainfall is an important parameter for disaster prevention and mitigation, which largely reflects the trend of disaster occurrence. Rainfall has an important impact on agricultural production, soil and water flow and engineering application. Accurate prediction of rainfall in a region can help agriculture. Water conservancy departments to improve the ability to prevent drought and waterlogging disasters, the harm to the minimum. With the frequent flood and waterlogging disasters in China in recent years, how to accurately and timely use meteorological data to forecast rainfall has become more and more important. The arrival of big data era, also brought new challenge to meteorological forecast industry. Weather data are mainly derived from ground observation, meteorological satellite remote sensing, weather radar and numerical forecast products. These four types of data account for more than 90% of the total data and are directly used in meteorological operations, weather forecasting, climate prediction and meteorological services. Stream data is a set of digitally encoded and continuous signals. In general, data flow can be regarded as an infinite and extensive application in network public opinion analysis, stock market trend, satellite positioning, financial real-time monitoring, Internet of things monitoring and real-time meteorological monitoring and so on. There is still much room for development in the field of rainfall prediction based on large-scale meteorological flow data. For the traditional rainfall prediction, the off-line meteorological data are often used to carry out batch training with the method of machine learning, that is, the learning process will not continue after all the training samples have been studied at one time. However, in practical applications, all samples in the training sample space can not be obtained at one time, but often in the order of time. Although large scale cluster can alleviate the problem of insufficient computing power caused by large amount of data to a certain extent, but for the newly arrived data, it is unable to process quickly and update the knowledge acquired by learning in time. In order to solve the problem of real-time calculation and massive processing of meteorological data, this paper presents a model of rainfall prediction based on online sequence based on Storm platform for extreme learning machine. The main contents and innovations of this paper are as follows: (1) aiming at the problem that the off-line batch forecasting method of meteorological data can not reflect the change of rainfall in time, a rainfall prediction model based on on-line sequence limit learning machine is proposed. Aiming at the large-scale and real-time characteristics of meteorological data, the algorithm of extreme learning machine is optimized on line. The model initializes several online extreme learning machine models. When the data of new batches are continuously reached, the model can continue to learn new samples on the basis of the existing training results. The method of random gradient descent and the adjustment of error weight are introduced to give error feedback to the new prediction results and update the error weight parameters in real time to improve the prediction accuracy of the model. (2) aiming at the problem of the massive high dimensional characteristics of meteorological data, In the stage of data preprocessing, the correlation coefficient between the decision attributes is used to analyze the meteorological data, and the correlation coefficient is used to filter the prediction attributes, which reduces the complexity of meteorological data and improves the efficiency of model training. In addition, Storm streaming big data frame and Kafka distributed message queue are used to train large scale meteorological data in parallel. Experimental results show that the algorithm runs on Storm platform and has excellent parallel performance and prediction accuracy.
【學(xué)位授予單位】：湘潭大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：P457.6;TP181

【參考文獻】

相關(guān)期刊論文 前10條

1 李志杰;李元香;王峰;何國良;匡立;;面向大數(shù)據(jù)分析的在線學(xué)習算法綜述[J];計算機研究與發(fā)展;2015年08期

2 孟小峰;慈祥;;大數(shù)據(jù)管理:概念、技術(shù)與挑戰(zhàn)[J];計算機研究與發(fā)展;2013年01期

3 姜文瑞;王玉英;郝小琪;李富鵬;;決策樹方法在氣溫預(yù)測中的應(yīng)用[J];計算機應(yīng)用與軟件;2012年08期

4 肖偉平;何宏;;基于遺傳算法的數(shù)據(jù)挖掘方法及應(yīng)用[J];湖南科技大學(xué)學(xué)報(自然科學(xué)版);2009年03期

5 鄒文安;劉立博;王鳳;;人工神經(jīng)網(wǎng)絡(luò)BP模型在枯季徑流量預(yù)測中的應(yīng)用[J];水資源研究;2008年03期

6 樊改娥;張順利;;淺談氣象預(yù)報的作用[J];科技情報開發(fā)與經(jīng)濟;2008年16期

7 石揚;張燕平;趙姝;張玲;田福生;汪小寒;;基于商空間的氣象時間序列數(shù)據(jù)挖掘研究[J];計算機工程與應(yīng)用;2007年01期

8 焦飛;黃天文;何華慶;;數(shù)據(jù)挖掘技術(shù)在氣溫長期變化趨勢預(yù)測中的應(yīng)用[J];廣東氣象;2006年02期

9 吳成東;許可;韓中華;裴濤;;基于粗糙集和決策樹的數(shù)據(jù)挖掘方法[J];東北大學(xué)學(xué)報;2006年05期

10 金龍,金健,姚才;A Short-Term Climate Prediction Model Based on a Modular Fuzzy Neural Network[J];Advances in Atmospheric Sciences;2005年03期

，

本文編號：2349609