【摘要】：近年來(lái),國(guó)內(nèi)霧霾天氣頻發(fā),范圍廣,時(shí)間長(zhǎng),嚴(yán)重影響民眾的身體健康,同時(shí)也對(duì)交通、電力和農(nóng)業(yè)造成的較大威脅,霧霾的治理已經(jīng)引起了政府和社會(huì)的高度關(guān)注。但由于各大城市空氣污染情況各異,且受到地理位置、氣象條件、工業(yè)成分、城市格局等因素影響,因此治理環(huán)境污染必須對(duì)城市污染來(lái)源進(jìn)行定性定量的科學(xué)研究,從而制定有明顯針對(duì)性的防治措施。大氣顆粒物的監(jiān)測(cè)與分析是了解空氣質(zhì)量的重要手段,而傳統(tǒng)的大氣顆粒物分析主要依靠顆粒物總體分析技術(shù)、人工識(shí)別顆粒物類別和來(lái)源解析,這些技術(shù)手段有明顯的缺陷:⑴傳統(tǒng)的顆粒物總體分析法無(wú)法反映顆粒物內(nèi)部特征,而現(xiàn)行基于單顆粒質(zhì)譜儀的分析技術(shù)則更加科學(xué);⑵單顆粒質(zhì)譜儀每天可采集數(shù)GB數(shù)據(jù),日積月累,數(shù)據(jù)量龐大,且顆粒物數(shù)據(jù)呈現(xiàn)半結(jié)構(gòu)化特點(diǎn),傳統(tǒng)的關(guān)系型數(shù)據(jù)庫(kù)不適用于本場(chǎng)景;⑶傳統(tǒng)的人工分析手段耗時(shí)長(zhǎng)、人工成本高、準(zhǔn)確率低,在面臨大數(shù)據(jù)量時(shí)無(wú)能為力,亟待一種顆粒物自動(dòng)分析技術(shù)。本文針對(duì)傳統(tǒng)大氣顆粒物分析技術(shù)的缺陷,設(shè)計(jì)了一種基于實(shí)時(shí)存儲(chǔ)技術(shù)的海量大氣顆粒物在線分析系統(tǒng),該系統(tǒng)由兩個(gè)子系統(tǒng)組成,分別是基于Google levelDB存儲(chǔ)引擎的海量數(shù)據(jù)存儲(chǔ)子系統(tǒng)RyDB和基于數(shù)據(jù)挖掘的在線分析子系統(tǒng)。底層數(shù)據(jù)存儲(chǔ)系統(tǒng)RyDB是一種KV型NoSQL數(shù)據(jù)庫(kù),采用levelDB存儲(chǔ)引擎,支持主從復(fù)制和集群部署,用于存儲(chǔ)實(shí)時(shí)采集或者離線收集的大氣顆粒物數(shù)據(jù);上層的在線分析系統(tǒng)采用自適應(yīng)諧振理論(ART)網(wǎng)絡(luò)聚類和邏輯回歸分類等數(shù)據(jù)挖掘技術(shù),實(shí)現(xiàn)對(duì)顆粒物數(shù)據(jù)的分類統(tǒng)計(jì)、來(lái)源解析等功能。經(jīng)過(guò)實(shí)驗(yàn)測(cè)試,數(shù)據(jù)存儲(chǔ)系統(tǒng)RyDB性能優(yōu)異,在測(cè)試環(huán)境中每秒讀寫能達(dá)10萬(wàn)次,具有高吞吐、低時(shí)延的特點(diǎn),能滿足實(shí)時(shí)存儲(chǔ)的需求;顆粒物在線分析系統(tǒng)的實(shí)驗(yàn)表明,系統(tǒng)時(shí)效性較強(qiáng),32萬(wàn)組顆粒物能夠在兩小時(shí)內(nèi)分析完畢,顆粒物分類的精確度為80%以上,滿足系統(tǒng)需求,實(shí)現(xiàn)顆粒物數(shù)據(jù)的自動(dòng)化分析。
[Abstract]:In recent years, domestic haze weather frequently, the scope is wide, the time is long, seriously affects the people's health, but also causes the bigger threat to the transportation, the electric power and the agriculture, the haze management has already caused the government and the society to pay close attention. However, due to the fact that the situation of air pollution in major cities is different and affected by the factors such as geographical location, meteorological conditions, industrial composition and urban pattern, it is necessary to carry out qualitative and quantitative scientific research on the sources of urban pollution in order to control the environmental pollution. Therefore, the prevention and cure measures have obvious pertinence. The monitoring and analysis of atmospheric particulate matter is an important means to understand the air quality, while the traditional analysis of atmospheric particulate matter mainly depends on the overall analysis technology of particulate matter, and manually identifies the category and source of particulate matter. These techniques have obvious defects. The traditional particle mass analysis method can not reflect the internal characteristics of particles, but the current analysis technology based on single particle mass spectrometer is more scientific and can collect GB data per day. The traditional relational database is not suitable for the traditional manual analysis method of this scene, which takes a long time, high labor cost and low accuracy. In the face of large amount of data, there is no way to analyze particulate matter automatically. Aiming at the defects of traditional atmospheric particulate matter analysis technology, this paper designs an on-line analysis system of massive atmospheric particulate matter based on real-time storage technology. The system consists of two subsystems. It is a mass data storage subsystem RyDB based on Google levelDB storage engine and an online analysis subsystem based on data mining. The underlying data storage system (RyDB) is a KV type NoSQL database, which uses levelDB storage engine to support master-slave replication and cluster deployment, and is used to store atmospheric particulate data collected in real time or offline. Data mining techniques such as adaptive resonance theory (ART) network clustering and logical regression classification are used in the upper layer online analysis system to realize the classification statistics and source analysis of particulate matter data. The experimental results show that the data storage system RyDB has excellent performance, can read and write 100000 times per second in the test environment, has the characteristics of high throughput and low delay, and can meet the demand of real-time storage. 320000 groups of particles can be analyzed in two hours, and the accuracy of classification of particulate matter is more than 80%, which meets the requirement of the system and realizes the automatic analysis of particulate matter data.
【學(xué)位授予單位】：中國(guó)科學(xué)院大學(xué)(中國(guó)科學(xué)院沈陽(yáng)計(jì)算技術(shù)研究所)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 王康;李東靜;陳海光;;分布式存儲(chǔ)系統(tǒng)中改進(jìn)的一致性哈希算法[J];計(jì)算機(jī)技術(shù)與發(fā)展;2016年07期

2 胡局新;張功杰;;基于K折交叉驗(yàn)證的選擇性集成分類算法[J];科技通報(bào);2013年12期

3 方錦清;;大數(shù)據(jù)浪潮沖擊下網(wǎng)絡(luò)科學(xué)與工程面臨的挑戰(zhàn)與機(jī)遇[J];自然雜志;2013年05期

4 陳明星;;城市化與經(jīng)濟(jì)發(fā)展關(guān)系的研究綜述[J];城市發(fā)展研究;2013年08期

5 申德榮;于戈;王習(xí)特;聶鐵錚;寇月;;支持大數(shù)據(jù)管理的NoSQL系統(tǒng)研究綜述[J];軟件學(xué)報(bào);2013年08期

6 崔杰;李陶深;蘭紅星;;基于Hadoop的海量數(shù)據(jù)存儲(chǔ)平臺(tái)設(shè)計(jì)與開發(fā)[J];計(jì)算機(jī)研究與發(fā)展;2012年S1期

7 楊錦;李肯立;吳帆;;異構(gòu)分布式系統(tǒng)的負(fù)載均衡調(diào)度算法[J];計(jì)算機(jī)工程;2012年02期

8 張琳;陳燕;汲業(yè);張金松;;一種基于密度的K-means算法研究[J];計(jì)算機(jī)應(yīng)用研究;2011年11期

9 銀燕;童堯青;魏玉香;王體健;李嘉鵬;楊衛(wèi)芬;樊曙先;;南京市大氣細(xì)顆粒物化學(xué)成分分析[J];大氣科學(xué)學(xué)報(bào);2009年06期

10 賀玲;吳玲達(dá);蔡益朝;;數(shù)據(jù)挖掘中的聚類算法綜述[J];計(jì)算機(jī)應(yīng)用研究;2007年01期

相關(guān)博士學(xué)位論文 前3條

1 楊英儀;面向云存儲(chǔ)副本復(fù)制的一致性關(guān)鍵技術(shù)研究[D];華南理工大學(xué);2015年

2 李磊;單顆粒氣溶膠質(zhì)譜儀的改進(jìn)及環(huán)境應(yīng)用[D];上海大學(xué);2014年

3 劉應(yīng)波;太陽(yáng)望遠(yuǎn)鏡海量數(shù)據(jù)存儲(chǔ)關(guān)鍵技術(shù)研究[D];中國(guó)科學(xué)院研究生院（云南天文臺(tái)）;2014年

相關(guān)碩士學(xué)位論文 前9條

1 楊成閣;貴陽(yáng)市PM_(10)、PM_(2.5)及其中多環(huán)芳烴的污染特征與來(lái)源解析研究[D];貴州師范大學(xué);2014年

2 張鵬翔;云計(jì)算下基于SVM的沙塵暴數(shù)據(jù)挖掘研究[D];內(nèi)蒙古工業(yè)大學(xué);2014年

3 余駿;面向海量天文數(shù)據(jù)的分布式存儲(chǔ)引擎的研究[D];天津大學(xué);2014年

4 張莉;基于單顆粒氣溶膠質(zhì)譜信息的分類方法研究及其應(yīng)用[D];上海大學(xué);2013年

5 李旭;五種決策樹算法的比較研究[D];大連理工大學(xué);2011年

6 楊宸鑄;基于HADOOP的數(shù)據(jù)挖掘研究[D];重慶大學(xué);2010年

7 王丹;遼寧省大氣環(huán)境監(jiān)測(cè)數(shù)據(jù)分析系統(tǒng)研究[D];東北大學(xué)　;2009年

8 韋德志;城市區(qū)域空氣質(zhì)量的BP神經(jīng)網(wǎng)絡(luò)預(yù)測(cè)研究[D];華中科技大學(xué);2009年

9 華敏潔;大氣環(huán)境質(zhì)量模型和GIS結(jié)合的研究[D];上海師范大學(xué);2005年

，

本文編號(hào)：2244048