基于HBase的矢量空間數(shù)據(jù)存取關鍵技術研究
發(fā)布時間：2018-07-23 14:36
【摘要】：隨著信息技術和空間信息獲取技術的發(fā)展、全球信息化推進和GIS(地理信息系統(tǒng))的廣泛應用,空間數(shù)據(jù)高速增長。面對日益增長的海量空間數(shù)據(jù),傳統(tǒng)的空間數(shù)據(jù)管理方案面臨高并發(fā)讀寫以及擴展性等等瓶頸。而云計算高擴展的存儲能力以及強大的計算能力則可以滿足海量數(shù)據(jù)存儲、大數(shù)據(jù)并行處理、高并發(fā)檢索等方面的需求。鑒于云計算技術的諸多優(yōu)點,論文針對如何利用云計算技術實現(xiàn)對海量矢量空間數(shù)據(jù)的存取展開研究。重點對云平臺下矢量空間數(shù)據(jù)的存儲模型、空間索引構(gòu)建、數(shù)據(jù)組織方案、數(shù)據(jù)的導入、空間查詢策略以及在HBase上進行屬性SQL查詢等進行了研究與設計。論文圍繞以下幾個方面開展工作:(1)矢量空間數(shù)據(jù)云存儲與檢索研究背景介紹及相關理論技術分析。論文闡述了海量空間數(shù)據(jù)存儲云存取的研究背景以及意義;分析了當前國內(nèi)外云計算概況及空間數(shù)據(jù)云存取的研究現(xiàn)狀以及當前研究的不足;結(jié)合Map Reduce并行計算框架特性深入分析了Map Reduce的矢量空間數(shù)據(jù)并行處理可行性,并探討分布式數(shù)據(jù)庫HBase以及SQL On Hadoop相關云計算技術存儲和管理海量矢量空間數(shù)據(jù)的優(yōu)勢。(2)構(gòu)建了基于HBase的矢量空間數(shù)據(jù)存儲模型以及No SQL模型與關系模型一體化的矢量空間數(shù)據(jù)的管理方案。針對矢量空間數(shù)據(jù)的特點,結(jié)合HBase數(shù)據(jù)模型,設計了矢量空間數(shù)據(jù)存儲模型,并采用四叉樹層次剖分技術設計了多級格網(wǎng)索引;結(jié)合空間信息多級格網(wǎng)編碼和Hilbert空間填充曲線的聚類特性,設計了符合HBase數(shù)據(jù)庫Row Key存儲規(guī)則的矢量空間數(shù)據(jù)標識編碼;根據(jù)HBase數(shù)據(jù)庫存儲規(guī)則以及Phoenix操作結(jié)構(gòu)化數(shù)據(jù)特性,提出并設計了No SQL模型與關系模型一體化的矢量空間數(shù)據(jù)的管理方案。(3)設計了矢量空間數(shù)據(jù)入庫以及并行構(gòu)建空間索引策略。結(jié)合Map Reduce并行處理特性討論并設計了單機導入和基于Map Reduce并行處理矢量空間數(shù)據(jù)的入庫方案以及基于Map Reduce設計了并行構(gòu)建空間索引方案。(4)根據(jù)多級網(wǎng)格索引策略設計了空間查詢策略。根據(jù)不同空間查詢算子、多級網(wǎng)格索引特點以及HBase掃描查詢數(shù)據(jù)特性,設計并實現(xiàn)了空間查詢算子優(yōu)化策略、合并網(wǎng)格編碼優(yōu)化查詢策略以及限制掃描列簇優(yōu)化數(shù)據(jù)過濾策略等三種空間查詢優(yōu)化策略。最后設計并實現(xiàn)了基于HBase的矢量空間數(shù)據(jù)存取原型系統(tǒng),實現(xiàn)了網(wǎng)格索引以及多級網(wǎng)格索引,通過網(wǎng)格索引與多級網(wǎng)格索引空間查詢效率對比實驗,驗證了多級網(wǎng)格索引的有效性。并基于多級網(wǎng)格索引,驗證了空間查詢算子優(yōu)化策略、合并網(wǎng)格編碼優(yōu)化查詢策略以及限制掃描列簇優(yōu)化數(shù)據(jù)過濾策略等三種空間查詢優(yōu)化策略的有效性。
[Abstract]:With the development of information technology and spatial information acquisition technology, the development of global information technology and the wide application of GIS (Geographic Information system), spatial data is growing rapidly. In the face of the increasing amount of spatial data, the traditional spatial data management scheme faces the bottleneck of high concurrent reading and writing and expansibility. Cloud computing can meet the needs of massive data storage, big data parallel processing, high concurrent retrieval and so on. In view of the many advantages of cloud computing technology, this paper focuses on how to use cloud computing technology to access mass vector space data. This paper focuses on the research and design of vector spatial data storage model, spatial index construction, data organization scheme, data import, spatial query strategy and attribute SQL query on HBase. This paper focuses on the following aspects: (1) introduction to the research background of vector spatial data cloud storage and retrieval and analysis of related theory and technology. This paper describes the research background and significance of cloud access for massive spatial data storage, analyzes the general situation of cloud computing at home and abroad, the research status quo of cloud access of spatial data and the shortcomings of current research. Combined with the characteristics of Map Reduce parallel computing framework, the feasibility of vector spatial data parallel processing in Map Reduce is analyzed. The advantages of distributed database HBase and SQL On Hadoop related cloud computing technology in storing and managing massive vector spatial data are discussed. (2) the vector spatial data storage model based on HBase and the integration of No SQL model and relational model are constructed. Vector spatial data management scheme. According to the characteristics of vector spatial data, combined with the HBase data model, the vector spatial data storage model is designed, and the multilevel grid index is designed by using the quadtree hierarchical partition technology. Combined with the clustering characteristics of spatial information multilevel grid coding and Hilbert space filling curve, the vector spatial data identification coding is designed according to HBase database Row Key storage rules, according to HBase database storage rules and Phoenix operation structured data characteristics. This paper proposes and designs a vector spatial data management scheme which integrates No SQL model and relational model. (3) A vector spatial data storage strategy and a parallel spatial index strategy are designed. Combined with the characteristics of Map Reduce parallel processing, this paper discusses and designs the input scheme of single machine importing vector spatial data and Map Reduce parallel processing vector spatial data, and designs a parallel spatial index scheme based on Map Reduce. (4) according to the multi-level grid index strategy, we design a parallel spatial index scheme. Spatial query strategy is designed. According to the characteristics of different spatial query operators, multilevel grid index and HBase scanning query data, the optimization strategy of spatial query operator is designed and implemented. There are three spatial query optimization strategies: merging trellis coding optimizing query strategy and restricting scanning column cluster optimizing data filtering strategy. Finally, the prototype system of vector spatial data access based on HBase is designed and implemented. The grid index and multilevel grid index are implemented. The efficiency of spatial query between grid index and multilevel grid index is compared. The validity of multilevel grid index is verified. Based on the multilevel grid index, the effectiveness of three spatial query optimization strategies, namely spatial query operator optimization strategy, combined grid coding optimization query strategy and restricted scan column cluster optimization data filtering strategy, is verified.
【學位授予單位】：江西理工大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：P208
本文編號：2139716
[Abstract]:With the development of information technology and spatial information acquisition technology, the development of global information technology and the wide application of GIS (Geographic Information system), spatial data is growing rapidly. In the face of the increasing amount of spatial data, the traditional spatial data management scheme faces the bottleneck of high concurrent reading and writing and expansibility. Cloud computing can meet the needs of massive data storage, big data parallel processing, high concurrent retrieval and so on. In view of the many advantages of cloud computing technology, this paper focuses on how to use cloud computing technology to access mass vector space data. This paper focuses on the research and design of vector spatial data storage model, spatial index construction, data organization scheme, data import, spatial query strategy and attribute SQL query on HBase. This paper focuses on the following aspects: (1) introduction to the research background of vector spatial data cloud storage and retrieval and analysis of related theory and technology. This paper describes the research background and significance of cloud access for massive spatial data storage, analyzes the general situation of cloud computing at home and abroad, the research status quo of cloud access of spatial data and the shortcomings of current research. Combined with the characteristics of Map Reduce parallel computing framework, the feasibility of vector spatial data parallel processing in Map Reduce is analyzed. The advantages of distributed database HBase and SQL On Hadoop related cloud computing technology in storing and managing massive vector spatial data are discussed. (2) the vector spatial data storage model based on HBase and the integration of No SQL model and relational model are constructed. Vector spatial data management scheme. According to the characteristics of vector spatial data, combined with the HBase data model, the vector spatial data storage model is designed, and the multilevel grid index is designed by using the quadtree hierarchical partition technology. Combined with the clustering characteristics of spatial information multilevel grid coding and Hilbert space filling curve, the vector spatial data identification coding is designed according to HBase database Row Key storage rules, according to HBase database storage rules and Phoenix operation structured data characteristics. This paper proposes and designs a vector spatial data management scheme which integrates No SQL model and relational model. (3) A vector spatial data storage strategy and a parallel spatial index strategy are designed. Combined with the characteristics of Map Reduce parallel processing, this paper discusses and designs the input scheme of single machine importing vector spatial data and Map Reduce parallel processing vector spatial data, and designs a parallel spatial index scheme based on Map Reduce. (4) according to the multi-level grid index strategy, we design a parallel spatial index scheme. Spatial query strategy is designed. According to the characteristics of different spatial query operators, multilevel grid index and HBase scanning query data, the optimization strategy of spatial query operator is designed and implemented. There are three spatial query optimization strategies: merging trellis coding optimizing query strategy and restricting scanning column cluster optimizing data filtering strategy. Finally, the prototype system of vector spatial data access based on HBase is designed and implemented. The grid index and multilevel grid index are implemented. The efficiency of spatial query between grid index and multilevel grid index is compared. The validity of multilevel grid index is verified. Based on the multilevel grid index, the effectiveness of three spatial query optimization strategies, namely spatial query operator optimization strategy, combined grid coding optimization query strategy and restricted scan column cluster optimization data filtering strategy, is verified.
【學位授予單位】：江西理工大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：P208
【參考文獻】
相關期刊論文 前10條
1 吳黎兵;邱鑫;葉璐瑤;王曉棟;聶雷;;基于Hadoop的SQL查詢引擎性能研究[J];華中師范大學學報(自然科學版);2016年02期
2 李德仁;;展望大數(shù)據(jù)時代的地球空間信息學[J];測繪學報;2016年04期
3 張葉;許國艷;花青;;基于HBase的矢量空間數(shù)據(jù)存儲與訪問優(yōu)化[J];計算機應用;2015年11期
4 宋寶燕;王俊陸;王妍;;基于范德蒙碼的HDFS優(yōu)化存儲策略研究[J];計算機學報;2015年09期
5 鄭坤;付艷麗;;基于HBase和GeoTools的矢量空間數(shù)據(jù)存儲模型研究[J];計算機應用與軟件;2015年03期
6 李清泉;李德仁;;大數(shù)據(jù)GIS[J];武漢大學學報(信息科學版);2014年06期
7 孟輝;朱美正;張鋒葉;;基于Hadoop的矢量空間數(shù)據(jù)庫技術[J];計算機與現(xiàn)代化;2014年02期
8 尹芳;馮敏;諸云強;劉睿;;基于開源Hadoop的矢量空間數(shù)據(jù)分布式處理研究[J];計算機工程與應用;2013年16期
9 陳崇成;林劍峰;吳小竹;巫建偉;連惠群;;基于NoSQL的海量空間數(shù)據(jù)云存儲與服務方法[J];地球信息科學學報;2013年02期
10 林德根;梁勤歐;;云GIS的內(nèi)涵與研究進展[J];地理科學進展;2012年11期
相關博士學位論文 前1條
1 范建永;基于Hadoop的云GIS若干關鍵技術研究[D];解放軍信息工程大學;2013年
相關碩士學位論文 前2條
1 祝若鑫;云計算環(huán)境下的空間矢量數(shù)據(jù)存儲與管理[D];解放軍信息工程大學;2015年
2 丁琛;基于HBase的空間數(shù)據(jù)分布式存儲和并行查詢算法研究[D];南京師范大學;2014年
，本文編號：2139716
本文鏈接：http://sikaile.net/kejilunwen/dizhicehuilunwen/2139716.html
最近更新
教材專著
