分布式環(huán)境下海量空間數(shù)據(jù)的存儲(chǔ)和并行查詢技術(shù)研究
本文選題:海量空間數(shù)據(jù) + 分布式存儲(chǔ); 參考:《江西理工大學(xué)》2012年碩士論文
【摘要】:隨著GIS在各行業(yè)的廣泛應(yīng)用,加上人們對(duì)空間數(shù)據(jù)的精度要求也不斷在增加,,于是GIS中需要管理的空間數(shù)據(jù)越來越多,數(shù)據(jù)量也越來越大近年來,分布式存儲(chǔ)和并行計(jì)算的不斷發(fā)展為海量空間數(shù)據(jù)的存儲(chǔ)和處理提供了一個(gè)新的方向然而,目前在分布式環(huán)境下對(duì)于海量空間數(shù)據(jù)的存儲(chǔ)和處理大部分是基于傳統(tǒng)關(guān)系數(shù)據(jù)庫,對(duì)于二三維的海量空間數(shù)據(jù)存儲(chǔ)效果不理想,其獨(dú)有的關(guān)系模型束縛對(duì)海量空間數(shù)據(jù)的快速訪問和處理能力因此,分布式環(huán)境下如何利用非關(guān)系數(shù)據(jù)庫和并行處理技術(shù)實(shí)現(xiàn)海量空間數(shù)據(jù)的高效存儲(chǔ)和快速處理具有重要的研究意義 首先比較分析目前幾類典型非關(guān)系數(shù)據(jù)庫的特點(diǎn),提出了非關(guān)系數(shù)據(jù)庫MongoDB的對(duì)于海量數(shù)據(jù)存儲(chǔ)優(yōu)勢(shì)在總結(jié)空間數(shù)據(jù)的特點(diǎn)基礎(chǔ)上,設(shè)計(jì)了海量空間數(shù)據(jù)在MongoDB中的存儲(chǔ)方式為了驗(yàn)證MongoDB對(duì)于海量數(shù)據(jù)存儲(chǔ)的可行性,與關(guān)系型數(shù)據(jù)庫SQLSERVER進(jìn)行了數(shù)據(jù)插入查詢實(shí)驗(yàn)比較接著根據(jù)上述的存儲(chǔ)方式,結(jié)合了并行處理框架Hadoop的MapReduce實(shí)現(xiàn)海量空間數(shù)據(jù)的并行處理最后設(shè)計(jì)實(shí)現(xiàn)了基于Hadoop+MongoDB的HMGIS數(shù)據(jù)庫管理器,并在多臺(tái)Linux服務(wù)器上搭建了HMGIS的分布式環(huán)境,通過利用HMGIS進(jìn)行海量空間數(shù)據(jù)導(dǎo)入和多種空間查詢等并行處理實(shí)驗(yàn)證明HMGIS的可行性和高效性 主要取得的研究成果: τ1υ利用MongoDB來對(duì)非結(jié)構(gòu)化的海量空間數(shù)據(jù)存儲(chǔ)是可行的; τ2υ利用Hadoop的MapReduce結(jié)合MongoDB數(shù)據(jù)庫實(shí)現(xiàn)的HMGIS數(shù)據(jù)庫管理器在海量空間數(shù)據(jù)的多種查詢方面比傳統(tǒng)關(guān)系數(shù)據(jù)庫具有更高的訪問和查詢效率; τ3υHMGIS設(shè)計(jì)實(shí)現(xiàn)可以進(jìn)一步為海量空間數(shù)據(jù)的分析和復(fù)雜的地理計(jì)算提供軟件支持
[Abstract]:With the wide application of GIS in various industries and the increasing demand for spatial data accuracy, more and more spatial data need to be managed in GIS, and the amount of data is becoming larger and larger in recent years. The continuous development of distributed storage and parallel computing provides a new direction for the storage and processing of massive space data. At present, most of the storage and processing of massive spatial data in distributed environment is based on the traditional relational database, which is not ideal for two or three dimensional mass spatial data storage, and its unique relational model restraints the rapid access and processing power of massive spatial data, and how to use non relation under the distributed environment. Database and parallel processing technology are of great significance for efficient storage and rapid processing of massive spatial data.
First, the characteristics of several typical non relational databases are compared and analyzed. On the basis of the characteristics of massive data storage in the non relational database MongoDB, the storage mode of massive spatial data in MongoDB is designed to verify the feasibility of MongoDB for mass data storage and the number of relational types. According to the storage mode of data insertion and query according to library SQLSERVER, the parallel processing of the parallel processing framework Hadoop MapReduce is combined to implement the parallel processing of massive spatial data. The HMGIS database manager based on Hadoop+MongoDB is designed and implemented, and a distributed loop of HMGIS is built on a number of Linux servers. The feasibility and efficiency of HMGIS are proved by parallel processing experiments such as massive spatial data import and multiple spatial queries using HMGIS.
The main achievements of the research are:
It is feasible for MongoDB 1 to store unstructured massive spatial data.
The HMGIS database manager, implemented by Hadoop MapReduce and MongoDB database, has higher access and query efficiency than traditional relational database in many queries of massive spatial data.
The design and implementation of HMGIS 3 can further provide software support for massive spatial data analysis and complex geographic computation.
【學(xué)位授予單位】:江西理工大學(xué)
【學(xué)位級(jí)別】:碩士
【學(xué)位授予年份】:2012
【分類號(hào)】:P208;TP333
【參考文獻(xiàn)】
相關(guān)期刊論文 前10條
1 廉燦國;李軍營;王長江;陳穎;;并行算法在彈道與諸元計(jì)算中的應(yīng)用[J];彈箭與制導(dǎo)學(xué)報(bào);2011年04期
2 程瑩;張?jiān)朴?房秉毅;徐雷;;云計(jì)算時(shí)代的數(shù)據(jù)庫研究[J];電信技術(shù);2011年01期
3 楊光祖;;基于分布式數(shù)據(jù)庫的病例系統(tǒng)設(shè)計(jì)與實(shí)現(xiàn)[J];大眾科技;2009年08期
4 李井杰;薛瑩;姚正明;;DFS在地理信息數(shù)字化生產(chǎn)管理中的應(yīng)用[J];測(cè)繪與空間地理信息;2010年04期
5 周長英;陳穎;;空間數(shù)據(jù)庫索引技術(shù)發(fā)展概況[J];黑龍江科技信息;2010年31期
6 郭本俊;王鵬;陳高云;黃健;;基于MPI的云計(jì)算模型[J];計(jì)算機(jī)工程;2009年24期
7 姚林;張永庫;;NoSQL的分布式存儲(chǔ)與擴(kuò)展解決方法[J];計(jì)算機(jī)工程;2012年06期
8 劉兵;郭福亮;;基于分布式數(shù)據(jù)庫系統(tǒng)查詢優(yōu)化策略的研究[J];計(jì)算機(jī)與數(shù)字工程;2005年12期
9 馮杭建,劉南,劉仁義;Linux環(huán)境下基于Oracle Spatial的分布式海量空間數(shù)據(jù)處理平臺(tái)的設(shè)計(jì)與實(shí)現(xiàn)[J];計(jì)算機(jī)應(yīng)用研究;2004年07期
10 劉金玲,曾學(xué)貴;地理信息系統(tǒng)Metadata共享和安全[J];中國安全科學(xué)學(xué)報(bào);2003年03期
本文編號(hào):1793159
本文鏈接:http://sikaile.net/kejilunwen/jisuanjikexuelunwen/1793159.html