基于Hadoop的云GIS若干關(guān)鍵技術(shù)研究
發(fā)布時間：2018-11-13 08:48
【摘要】：云計算是互聯(lián)網(wǎng)計算發(fā)展到一定階段的產(chǎn)物,是并行計算、網(wǎng)格計算等多種新型計算方式演進的最新結(jié)果。云計算無限擴展的存儲技術(shù)可以滿足快速增長的空間數(shù)據(jù)對存儲空間的需求,強大的計算能力可為空間信息的檢索、處理、分析等提供高速的服務(wù)保證。本文針對GIS當(dāng)前所面臨的海量數(shù)據(jù)存儲、處理、分析與持續(xù)服務(wù)等問題,結(jié)合GIS和云計算的特點,將開源Hadoop云計算平臺應(yīng)用到空間信息服務(wù)領(lǐng)域,研究利用Hadoop云計算平臺提供的分布式存儲能力和并行計算能力,構(gòu)建基于Hadoop的GIS應(yīng)用,并對其中的一些關(guān)鍵技術(shù)進行研究。本文主要工作如下:(1)在分析商業(yè)云GIS體系結(jié)構(gòu)的基礎(chǔ)上,設(shè)計了基于Hadoop的云GIS體系結(jié)構(gòu)。體系結(jié)構(gòu)包括物理設(shè)備層、平臺層、軟件層、應(yīng)用層等4層,以及橫跨多個層次的用戶管理、服務(wù)管理、資源管理、監(jiān)控系統(tǒng)、容災(zāi)備份、運營管理等服務(wù)。設(shè)計了基于Hadoop的云GIS部署模式。整個基于Hadoop的云GIS系統(tǒng)由平臺管理門戶、GIS Web服務(wù)器集群及多個Hadoop集群組成。分析了體系結(jié)構(gòu)特點,為后面的研究內(nèi)容奠定了基礎(chǔ)。(2)本文在空間信息格網(wǎng)單元和OGC簡單要素規(guī)范基礎(chǔ)上,結(jié)合矢量數(shù)據(jù)的特點,利用格網(wǎng)單元ID的唯一性、多尺度性及索引性,提出了一種以格網(wǎng)單元為存儲單位的矢量數(shù)據(jù)存儲方案;結(jié)合矢量要素的定性屬性數(shù)據(jù),設(shè)計了矢量數(shù)據(jù)的存儲格式“GWKT(Grid Well-know Text)”;為了達到矢量要素標(biāo)識全球唯一,本文基于格網(wǎng)單元和Hilbert曲線的Base16編碼,結(jié)合HBase數(shù)據(jù)庫的特點,設(shè)計了矢量要素標(biāo)識的編碼,并實現(xiàn)了編碼的生成算法;研究實現(xiàn)了基于單調(diào)鏈的矢量要素分割與合并算法,可有效的分割和合并線狀和面狀要素;在HBase基礎(chǔ)上,擴展了HBase的數(shù)據(jù)類型及其過濾器,實現(xiàn)了屬性數(shù)據(jù)的快速查詢。(3)針對海量空間數(shù)據(jù)處理能力不足問題,設(shè)計了基于HDFS的矢量數(shù)據(jù)存儲格式,實現(xiàn)了基于MapReduce的矢量數(shù)據(jù)分割入庫并行處理模型;在MapReduce數(shù)據(jù)過濾器的基礎(chǔ)上,設(shè)計了適合基于格網(wǎng)單元的空間數(shù)據(jù)并行計算模型,并以矢量數(shù)據(jù)緩沖區(qū)分析作為實例進行了驗證;設(shè)計實現(xiàn)了基于MapReduce的kNN空間數(shù)據(jù)查詢算法;分析了基于MapReduce的空間數(shù)據(jù)并行計算效率。(4)在空間信息服務(wù)方面,本文在OGC標(biāo)準(zhǔn)服務(wù)基礎(chǔ)上,對服務(wù)參數(shù)進行了擴展,設(shè)計了云GIS空間信息服務(wù)分層體系結(jié)構(gòu),實現(xiàn)了基于空間信息多級格網(wǎng)的WMS、WMTS、WFS和WPS等服務(wù);設(shè)計實現(xiàn)了空間信息服務(wù)接口,以實現(xiàn)客戶端與服務(wù)器端的完全解耦。(5)在前文研究基礎(chǔ)上,設(shè)計并實現(xiàn)了基于Hadoop的云GIS原型系統(tǒng),完成了海量柵格與矢量數(shù)據(jù)的高效存儲與管理、空間數(shù)據(jù)并行計算以及基于Hadoop的空間信息服務(wù)等關(guān)鍵模塊;并對相關(guān)模塊做了性能測試,驗證了本文提出的相關(guān)存儲模型和計算模型的可行性、有效性以及高效性。
[Abstract]:Cloud computing is the product of the development of Internet computing to a certain stage. It is the latest result of the evolution of many new computing methods, such as parallel computing, grid computing and so on. The storage technology of infinite expansion of cloud computing can meet the demand for storage space of rapidly growing spatial data. Powerful computing power can provide high-speed service guarantee for spatial information retrieval, processing, analysis and so on. Aiming at the problems of massive data storage, processing, analysis and continuous service in GIS, combining the characteristics of GIS and cloud computing, the open source Hadoop cloud computing platform is applied to the field of spatial information service. The distributed storage and parallel computing capabilities provided by Hadoop cloud computing platform are used to construct GIS applications based on Hadoop, and some key technologies are studied. The main work of this paper is as follows: (1) based on the analysis of business cloud GIS architecture, a cloud GIS architecture based on Hadoop is designed. The architecture includes four layers: physical device layer, platform layer, software layer, application layer, and so on, as well as services such as user management, service management, resource management, monitoring system, disaster recovery backup, operation management and so on. A cloud GIS deployment model based on Hadoop is designed. The whole cloud GIS system based on Hadoop consists of platform management portal, GIS Web server cluster and multiple Hadoop clusters. The characteristics of the architecture are analyzed, which lays the foundation for the later research. (2) based on the specification of spatial information grid element and OGC simple element, combined with the characteristics of vector data, the uniqueness of grid element ID is used in this paper. In this paper, a vector data storage scheme with grid unit as storage unit is proposed, which is multiscale and indexed. Combining the qualitative attribute data of vector elements, the storage format "GWKT (Grid Well-know Text" of vector data is designed. In order to achieve the global uniqueness of vector element identification, based on the Base16 coding of grid element and Hilbert curve, combined with the characteristics of HBase database, the encoding of vector element identification is designed, and the coding algorithm is realized. The algorithm of vector element segmentation and merging based on monotone chain is studied, which can effectively segment and merge linear and plane elements. On the basis of HBase, the data type and its filter of HBase are extended, and the fast query of attribute data is realized. (3) aiming at the problem of insufficient processing ability of massive spatial data, the vector data storage format based on HDFS is designed. The parallel processing model of vector data segmentation and input database based on MapReduce is implemented. On the basis of MapReduce data filter, the parallel computing model of spatial data based on grid element is designed, and the vector data buffer analysis is used as an example to verify it, and the kNN spatial data query algorithm based on MapReduce is designed and implemented. The efficiency of parallel computing of spatial data based on MapReduce is analyzed. (4) in the aspect of spatial information service, the service parameters are extended on the basis of OGC standard service, and the hierarchical architecture of cloud GIS spatial information service is designed. The services such as WMS,WMTS,WFS and WPS based on spatial information multilevel grid are realized. The spatial information service interface is designed and implemented to realize the complete decoupling between the client and the server. (5) based on the previous research, a cloud GIS prototype system based on Hadoop is designed and implemented. The key modules, such as efficient storage and management of massive grid and vector data, parallel computing of spatial data and spatial information service based on Hadoop, are completed. The performance tests of the related modules are carried out to verify the feasibility, effectiveness and efficiency of the related storage model and the computing model proposed in this paper.
【學(xué)位授予單位】：解放軍信息工程大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2013
【分類號】：P208
本文編號：2328605
[Abstract]:Cloud computing is the product of the development of Internet computing to a certain stage. It is the latest result of the evolution of many new computing methods, such as parallel computing, grid computing and so on. The storage technology of infinite expansion of cloud computing can meet the demand for storage space of rapidly growing spatial data. Powerful computing power can provide high-speed service guarantee for spatial information retrieval, processing, analysis and so on. Aiming at the problems of massive data storage, processing, analysis and continuous service in GIS, combining the characteristics of GIS and cloud computing, the open source Hadoop cloud computing platform is applied to the field of spatial information service. The distributed storage and parallel computing capabilities provided by Hadoop cloud computing platform are used to construct GIS applications based on Hadoop, and some key technologies are studied. The main work of this paper is as follows: (1) based on the analysis of business cloud GIS architecture, a cloud GIS architecture based on Hadoop is designed. The architecture includes four layers: physical device layer, platform layer, software layer, application layer, and so on, as well as services such as user management, service management, resource management, monitoring system, disaster recovery backup, operation management and so on. A cloud GIS deployment model based on Hadoop is designed. The whole cloud GIS system based on Hadoop consists of platform management portal, GIS Web server cluster and multiple Hadoop clusters. The characteristics of the architecture are analyzed, which lays the foundation for the later research. (2) based on the specification of spatial information grid element and OGC simple element, combined with the characteristics of vector data, the uniqueness of grid element ID is used in this paper. In this paper, a vector data storage scheme with grid unit as storage unit is proposed, which is multiscale and indexed. Combining the qualitative attribute data of vector elements, the storage format "GWKT (Grid Well-know Text" of vector data is designed. In order to achieve the global uniqueness of vector element identification, based on the Base16 coding of grid element and Hilbert curve, combined with the characteristics of HBase database, the encoding of vector element identification is designed, and the coding algorithm is realized. The algorithm of vector element segmentation and merging based on monotone chain is studied, which can effectively segment and merge linear and plane elements. On the basis of HBase, the data type and its filter of HBase are extended, and the fast query of attribute data is realized. (3) aiming at the problem of insufficient processing ability of massive spatial data, the vector data storage format based on HDFS is designed. The parallel processing model of vector data segmentation and input database based on MapReduce is implemented. On the basis of MapReduce data filter, the parallel computing model of spatial data based on grid element is designed, and the vector data buffer analysis is used as an example to verify it, and the kNN spatial data query algorithm based on MapReduce is designed and implemented. The efficiency of parallel computing of spatial data based on MapReduce is analyzed. (4) in the aspect of spatial information service, the service parameters are extended on the basis of OGC standard service, and the hierarchical architecture of cloud GIS spatial information service is designed. The services such as WMS,WMTS,WFS and WPS based on spatial information multilevel grid are realized. The spatial information service interface is designed and implemented to realize the complete decoupling between the client and the server. (5) based on the previous research, a cloud GIS prototype system based on Hadoop is designed and implemented. The key modules, such as efficient storage and management of massive grid and vector data, parallel computing of spatial data and spatial information service based on Hadoop, are completed. The performance tests of the related modules are carried out to verify the feasibility, effectiveness and efficiency of the related storage model and the computing model proposed in this paper.
【學(xué)位授予單位】：解放軍信息工程大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2013
【分類號】：P208
【參考文獻】
相關(guān)期刊論文 前10條
1 劉義;景寧;陳犖;熊偉;;MapReduce框架下基于R-樹的k-近鄰連接算法[J];軟件學(xué)報;2013年08期
2 陳崇成;林劍峰;吳小竹;巫建偉;連惠群;;基于NoSQL的海量空間數(shù)據(jù)云存儲與服務(wù)方法[J];地球信息科學(xué)學(xué)報;2013年02期
3 夏英;楊選倫;;云環(huán)境中基于金字塔模型的影像數(shù)據(jù)存儲方法[J];重慶郵電大學(xué)學(xué)報(自然科學(xué)版);2012年06期
4 李德仁;姚遠;邵振峰;;智慧地球時代測繪地理信息學(xué)的新使命[J];測繪科學(xué);2012年06期
5 林德根;梁勤歐;;云GIS的內(nèi)涵與研究進展[J];地理科學(xué)進展;2012年11期
6 范協(xié)裕;任應(yīng)超;楊崇俊;唐建智;易雄鷹;;基于集群技術(shù)的可伸縮云GIS服務(wù)平臺研究[J];計算機應(yīng)用研究;2012年10期
7 張廣弟;汪秀兵;胡亞磊;;基于hadoop的DEM格網(wǎng)建立研究[J];科技視界;2012年28期
8 茍麗美;朱美正;李艷明;;RESTful風(fēng)格地圖瓦片服務(wù)的研究[J];計算機工程與設(shè)計;2012年09期
9 王旭東;張福浩;張麗萍;;基于地形數(shù)據(jù)的瓦片金字塔構(gòu)建與組織[J];測繪與空間地理信息;2012年06期
10 張鵬飛;陳榮國;謝炯;張明波;;空間數(shù)據(jù)庫的WFS長事務(wù)管理方案研究[J];測繪科學(xué)技術(shù)學(xué)報;2012年03期
相關(guān)博士學(xué)位論文 前2條
1 康俊鋒;云計算環(huán)境下高分辨率遙感影像存儲與高效管理技術(shù)研究[D];浙江大學(xué);2011年
2 趙春宇;高性能并行GIS中矢量空間數(shù)據(jù)存取與處理關(guān)鍵技術(shù)研究[D];武漢大學(xué);2006年
相關(guān)碩士學(xué)位論文 前3條
1 楊柳;基于云計算的GIS應(yīng)用模式研究[D];河南大學(xué);2011年
2 霍樹民;基于Hadoop的海量影像數(shù)據(jù)管理關(guān)鍵技術(shù)研究[D];國防科學(xué)技術(shù)大學(xué);2010年
3 岳昌華;空間數(shù)據(jù)庫K最接近對查詢算法的研究[D];大連海事大學(xué);2010年
，本文編號：2328605
本文鏈接：http://sikaile.net/kejilunwen/dizhicehuilunwen/2328605.html
最近更新
教材專著
