本文選題：大數(shù)據(jù) + 排污監(jiān)管　； 參考：《江蘇大學(xué)》2017年碩士論文

【摘要】：隨著我國經(jīng)濟(jì)的快速發(fā)展,人民生活及工業(yè)用水量大幅提升,產(chǎn)生的污水排放也隨之增多,尤其是工業(yè)廢水的排放問題十分嚴(yán)重。對此,建立城市排污監(jiān)管系統(tǒng)能夠有效地提前發(fā)現(xiàn)問題并從源頭解決問題,是未來此類問題解決的必然趨勢。目前,污水產(chǎn)生的源頭分布十分廣泛、排放時間不固定,而且污水的監(jiān)測判定指標(biāo)很多,傳統(tǒng)解決方案存在采集周期長、評判指標(biāo)數(shù)量大且維度高、系統(tǒng)處理效率低等問題。針對上述問題,本課題在深入分析污水?dāng)?shù)據(jù)特性的基礎(chǔ)上,結(jié)合大數(shù)據(jù)處理技術(shù),研究并實現(xiàn)了一種基于大數(shù)據(jù)分析的城市排污監(jiān)管系統(tǒng)。本文的主要工作包括以下內(nèi)容:1)提出一種污水大數(shù)據(jù)的實時監(jiān)測技術(shù)。首先,基于集成學(xué)習(xí)Adaboost算法的思想,采用FNN_Adaboost模型來預(yù)測未知的復(fù)雜污水指標(biāo),提高對未知指標(biāo)值的預(yù)測精度。然后,在降低污水?dāng)?shù)據(jù)維度基礎(chǔ)上,通過改進(jìn)k-means聚類算法增強(qiáng)污水大數(shù)據(jù)的聚類效果。最后,基于Spark Streaming流式數(shù)據(jù)處理框架實現(xiàn)對工廠排污超標(biāo)實時監(jiān)測。2)設(shè)計一種污水大數(shù)據(jù)的存儲體系架構(gòu)。針對污水?dāng)?shù)據(jù)的數(shù)量大、多樣性等特征,采用主從式架構(gòu)搭建Hadoop集群,并在HBase數(shù)據(jù)庫中設(shè)計了面向列的污水?dāng)?shù)據(jù)存儲模型,不僅能存儲海量的污水?dāng)?shù)據(jù)信息,而且有效提高了數(shù)據(jù)處理效率。3)提出一種污水大數(shù)據(jù)的預(yù)測分析技術(shù)。針對工廠排污情況的差異性及污水?dāng)?shù)據(jù)的動態(tài)性,在進(jìn)行序列化處理與分析的基礎(chǔ)上,對比采用線性和非線性時間序列預(yù)測模型進(jìn)行數(shù)據(jù)預(yù)測的效果,并選取最優(yōu)模型來實現(xiàn)污水大數(shù)據(jù)的預(yù)測分析。最后通過實際應(yīng)用分析驗證了該技術(shù)的可行性和有效性。4)在對上述技術(shù)進(jìn)行研究的基礎(chǔ)上,本文設(shè)計并實現(xiàn)了一種基于大數(shù)據(jù)分析的城市排污監(jiān)管系統(tǒng)。該系統(tǒng)借鑒MVC的設(shè)計思想,實現(xiàn)了數(shù)據(jù)解析,數(shù)據(jù)分析處理,數(shù)據(jù)存儲,數(shù)據(jù)查詢,預(yù)測報警等功能模塊,不僅能夠?qū)Τ鞘泄S排污情況的全面智能管理,而且可以進(jìn)行大數(shù)據(jù)關(guān)聯(lián)分析,從而積極推進(jìn)城市環(huán)保大數(shù)據(jù)的建設(shè)和發(fā)展。
[Abstract]:With the rapid development of our country's economy, the amount of water used in people's life and industry has been greatly increased, and the sewage discharge has increased, especially the problem of industrial wastewater discharge is very serious. Therefore, it is an inevitable trend to set up urban sewage supervision system to find out the problem and solve it from the source effectively in the future. At present, the source of sewage is widely distributed, the discharge time is not fixed, and there are many monitoring and judging indexes of sewage. The traditional solution has many problems, such as long collection period, large number of evaluation indexes and high dimensionality, low efficiency of system treatment and so on. In view of the above problems, based on the in-depth analysis of the characteristics of sewage data, combined with big data treatment technology, this paper studies and implements a city sewage supervision system based on big data analysis. The main work of this paper includes the following contents: 1) A real-time monitoring technology for wastewater big data is proposed. Firstly, based on the idea of integrated learning Adaboost algorithm, FNN_Adaboost model is used to predict the unknown complex wastewater index, and the prediction accuracy of unknown index value is improved. Then, on the basis of reducing the dimension of sewage data, the improved k-means clustering algorithm is used to enhance the clustering effect of sewage big data. Finally, a storage system of wastewater big data is designed based on Spark Streaming streaming data processing framework. Aiming at the characteristics of large quantity and diversity of sewage data, Hadoop cluster is constructed by master-slave architecture, and a column-oriented sewage data storage model is designed in HBase database, which can not only store massive sewage data information. Moreover, the efficiency of data processing is improved. 3) A kind of forecast and analysis technology of sewage big data is put forward. In view of the difference of sewage discharge in factories and the dynamic nature of sewage data, on the basis of serialization and analysis, the effects of linear and nonlinear time series prediction models for data prediction are compared. And select the best model to realize the forecast and analysis of the sewage big data. Finally, the feasibility and effectiveness of the technology are verified by practical application analysis. On the basis of the research on the above technology, this paper designs and implements a kind of urban sewage supervision system based on big data analysis. Using the design idea of MVC for reference, the system realizes the function modules of data analysis, data analysis and processing, data storage, data query, prediction and alarm, etc. And can carry on big data relevance analysis, thus actively promote the construction and development of urban environmental protection big data.
【學(xué)位授予單位】：江蘇大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP311.13;TP311.52

【參考文獻(xiàn)】

相關(guān)期刊論文 前5條

1 王思揚(yáng);邱凌;王麗娟;唐小軍;;淺談大數(shù)據(jù)在環(huán)境保護(hù)中的應(yīng)用[J];四川環(huán)境;2016年05期

2 王建民;;《生態(tài)環(huán)境大數(shù)據(jù)建設(shè)總體方案》政策解讀[J];環(huán)境保護(hù);2016年14期

3 喬俊飛;李瑞祥;柴偉;韓紅桂;;基于PSO-ESN神經(jīng)網(wǎng)絡(luò)的污水BOD預(yù)測[J];控制工程;2016年04期

4 ;促進(jìn)大數(shù)據(jù)發(fā)展行動綱要[J];成組技術(shù)與生產(chǎn)現(xiàn)代化;2015年03期

5 俞新凱;曾光輝;毛敏;;一種基于MVC設(shè)計模式的開發(fā)框架[J];軟件導(dǎo)刊;2015年06期

相關(guān)碩士學(xué)位論文 前1條

1 邱菊;COD測量方法與快速監(jiān)測系統(tǒng)研究[D];浙江大學(xué);2003年

，

本文編號：1794670