【摘要】：Argo剖面浮標是目前唯一實時獲取全球上層海洋三維觀測數(shù)據(jù)的手段,其觀測的數(shù)據(jù)反映了海洋立體溫鹽分布的情況,對研究海洋環(huán)流、全球氣候變化、海洋分析預(yù)報系統(tǒng)提供了數(shù)據(jù)基礎(chǔ),具有極其重要的應(yīng)用意義與科學(xué)價值。本文以Argo剖面浮標數(shù)據(jù)作為海洋大數(shù)據(jù)信息安全研究的切入點,針對Argo剖面浮標數(shù)據(jù)受環(huán)境、設(shè)備等不確定因素的影響導(dǎo)致數(shù)據(jù)出現(xiàn)異常的問題,結(jié)合Argo剖面浮標數(shù)據(jù)具有數(shù)據(jù)量大、地域性、非線性分布、離散等特點,開展了Argo剖面浮標數(shù)據(jù)異常檢測方法的深入研究,為提高Argo剖面浮標數(shù)據(jù)準確性、可靠性提供理論依據(jù)和技術(shù)手段。本文針對異常檢測的訓(xùn)練階段與異常檢測階段分別展開分析與研究。首先,在訓(xùn)練階段,針對Argo剖面浮標數(shù)據(jù)文件格式復(fù)雜、數(shù)據(jù)量龐大的問題,提出基于MapReduce技術(shù)的Argo主剖面生成的AMPC(information fusion algorithm for Argo profile base on MapReduce and Principal Curves)算法。該算法采用MapReduce技術(shù)有效地提高執(zhí)行效率,另外,將剖面信息以經(jīng)緯度進行分類,增強剖面間關(guān)聯(lián)性,凸顯剖面地域特點,并在K-主曲線理論的基礎(chǔ)上,使用不斷添加擬合剖面點的方式生成主剖面,減少異常檢測階段的數(shù)據(jù)存儲量,為剖面可能存在的點異常、上下文異常、集合異常提供參考依據(jù)。其次,在異常檢測階段,汲取基于“三倍標準差”準則異常檢測方法與基于預(yù)測模型異常檢測方法的優(yōu)點,改進設(shè)計一種結(jié)合分段“三倍標準差”準則和k-近鄰子剖面與主曲線預(yù)測方法的基于自適應(yīng)異常閾值的異常檢測方法。該方法以訓(xùn)練階段生成的主剖面作為參考,綜合當前待測剖面點對主剖面的偏離情況與剖面隨深度變化的趨勢給異常檢測帶來的影響,動態(tài)計算各剖面點的異常閾值,進一步提高異常檢測方法的性能與檢測效果。驗證試驗表明:通過全球Argo剖面浮標數(shù)據(jù)的驗證,本文研究的針對Argo剖面浮標數(shù)據(jù)特點的異常檢測方法高效地結(jié)合歷史剖面數(shù)據(jù),有效地規(guī)避靜態(tài)閾值檢測的片面性,具有良好的異常檢測效果,異常檢測的準確度得到明顯提高。
[Abstract]:Argo profile buoy is the only real-time method to obtain three-dimensional observational data from global upper ocean. The observed data reflect the distribution of ocean temperature and salt in three-dimensional ocean. It is very important to study ocean circulation and global climate change. The ocean analysis and forecast system provides the basis of data, and has very important application significance and scientific value. This paper takes the Argo profile buoy data as the breakthrough point of the marine big data information security research, aiming at the problem that the Argo profile buoy data is influenced by the uncertain factors such as environment, equipment and so on, which leads to the abnormal data. According to the characteristics of Argo profile buoy data, such as large amount of data, region, non-linear distribution, discrete and so on, an in-depth study on anomaly detection method of Argo profile buoy data is carried out in order to improve the accuracy of Argo profile buoy data. Reliability provides theoretical basis and technical means. In this paper, the training phase and anomaly detection phase of anomaly detection are analyzed and studied respectively. Firstly, in the training stage, aiming at the complicated format and huge data volume of the Argo profile buoy data file, a AMPC (information fusion algorithm for Argo profile base on MapReduce and Principal Curves) algorithm based on MapReduce technology for the generation of Argo main profile is proposed. In addition, the section information is classified by latitude and longitude to enhance the correlation between sections and highlight the regional characteristics of the section, and on the basis of K-principal curve theory, the algorithm uses MapReduce technology to improve the efficiency of execution effectively. The main section is generated by continuously adding fitting section points to reduce the amount of data stored in the anomaly detection phase, which provides reference for possible point exceptions, context exceptions, and set exceptions in the profile. Secondly, in the anomaly detection phase, the advantages of the anomaly detection method based on the "triple standard deviation" criterion and the anomaly detection method based on predictive model are drawn. An improved anomaly detection method based on adaptive anomaly threshold is designed, which combines the segmented "triple standard deviation" criterion and the prediction method of k-nearest neighbor profile and principal curve. In this method, the main section generated during the training stage is taken as a reference, and the deviation of the current section point to the main section and the influence of the variation trend of the profile with the depth on the anomaly detection are integrated, and the abnormal threshold values of each section point are calculated dynamically. Further improve the performance and detection effect of the anomaly detection method. The verification test shows that: through the verification of global Argo profile buoy data, the anomaly detection method based on the characteristics of Argo profile buoy data studied in this paper effectively combines historical profile data, and effectively avoids the one-sidedness of static threshold detection. It has good effect of anomaly detection, and the accuracy of anomaly detection is improved obviously.
【學(xué)位授予單位】：桂林電子科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2016
【分類號】：P715.2;TP311.13

【相似文獻】

相關(guān)期刊論文 前10條

1 黃麗超;陳國弟;;自動站雨量數(shù)據(jù)異常的處理方法[J];現(xiàn)代農(nóng)業(yè)科技;2010年01期

2 朱石軍;馬廣慶;趙帥;;延慶臺五里營井氣氡數(shù)據(jù)異常分析[J];四川地震;2013年04期

3 陳慶慶;張漢平;;自動站整點上傳數(shù)據(jù)異常處理方法[J];陜西氣象;2012年01期

4 黎錦雷;黃春莎;;一體化數(shù)據(jù)信息分析反饋[J];氣象研究與應(yīng)用;2013年04期

5 李鵬飛;;深層地溫數(shù)據(jù)異常原因探討[J];農(nóng)業(yè)與技術(shù);2012年10期

6 莫小梅;林展新;蘇崇升;彭自強;;10cm地溫數(shù)據(jù)異常分析及儀器故障排除[J];氣象水文海洋儀器;2012年02期

7 劉巖;王宗海;王青利;;自動氣象站數(shù)據(jù)異�；蛉睖y的處理方法[J];現(xiàn)代農(nóng)業(yè)科技;2011年18期

8 宋明明;尹雪梅;李秀娟;;自動站雷擊后正點數(shù)據(jù)的處理[J];科技創(chuàng)新導(dǎo)報;2013年14期

9 陳明;樊新宇;;AMS-Ⅱ型自動氣象觀測系統(tǒng)降水數(shù)據(jù)異常情況的處理[J];氣象水文海洋儀器;2011年04期

10 張立清;張洪衛(wèi);;自動氣象站觀測數(shù)據(jù)異常的檢查方法[J];氣象水文海洋儀器;2010年03期

相關(guān)會議論文 前6條

1 蘇艷;胡少棟;胡映紅;;對自動氣象站數(shù)據(jù)異常原因及防范措施的探討[A];第七屆長三角氣象科技論壇論文集[C];2010年

2 華連生;溫華洋;徐光清;吳必文;汪臘寶;;“雙套站”氣溫模擬評估[A];第28屆中國氣象學(xué)會年會——S1第四屆氣象綜合探測技術(shù)研討會[C];2011年

3 秦榕;井立紅;何亞平;瑪依努爾·阿不拉;侯玲紅;;自動站凈全輻射異常值的處理方法[A];第26屆中國氣象學(xué)會年會第三屆氣象綜合探測技術(shù)研討會分會場論文集[C];2009年

4 許嘉玲;王超球;趙秀英;;自動氣象站數(shù)據(jù)異常的原因分析[A];2006年華南地區(qū)學(xué)術(shù)交流會論文集[C];2006年

5 陳璐;;北京地區(qū)自動氣象站數(shù)據(jù)異常因素分析[A];第28屆中國氣象學(xué)會年會——S1第四屆氣象綜合探測技術(shù)研討會[C];2011年

6 陳靜;張遠;張青珍;王國安;邢用書;;新氣象 新問題 新思路[A];第28屆中國氣象學(xué)會年會——S1第四屆氣象綜合探測技術(shù)研討會[C];2011年

相關(guān)重要報紙文章 前3條

1 東方證券首席經(jīng)濟學(xué)家 邵宇;三因素導(dǎo)致金融數(shù)據(jù)異常[N];第一財經(jīng)日報;2014年

2 本報記者 宋菁;兩項外貿(mào)數(shù)據(jù)異常[N];21世紀經(jīng)濟報道;2013年

3 本報記者 韓雪萌;結(jié)售匯數(shù)據(jù)異常 源于企業(yè)結(jié)匯動機減弱[N];金融時報;2014年

相關(guān)碩士學(xué)位論文 前1條

1 胡瑩;Argo剖面浮標數(shù)據(jù)異常檢測方法研究[D];桂林電子科技大學(xué);2016年

，

本文編號：2452088