基于集群的地理空間數(shù)據(jù)組織與訪問方法
發(fā)布時間：2018-07-29 12:46
【摘要】：隨著大數(shù)據(jù)時代的到來,地理信息系統(tǒng)領(lǐng)域面對的地理空間數(shù)據(jù)呈現(xiàn)出了數(shù)據(jù)量越來越大、數(shù)據(jù)精度越來越高、計算復(fù)雜度越來越大的特點,傳統(tǒng)的基于單機的地理信息系統(tǒng)已經(jīng)難以有效解決數(shù)據(jù)密集、計算密集的復(fù)雜地理計算問題。與此同時,多核、眾核、GPU、集群等新型計算硬件架構(gòu)飛速發(fā)展,性能不斷提高,成本不斷下降,為基于這些架構(gòu)設(shè)計實現(xiàn)高性能地理信息系統(tǒng)平臺提供了有效途徑。與傳統(tǒng)地理信息系統(tǒng)相比,高性能地理信息系統(tǒng)帶來了數(shù)據(jù)處理規(guī)模擴大、數(shù)據(jù)集成度提高、功能協(xié)同方便等諸多優(yōu)勢,有利于擴展應(yīng)用范圍,推動大規(guī)模地理信息應(yīng)用的發(fā)展。因此,基于新型硬件架構(gòu)開展高性能地理信息系統(tǒng)領(lǐng)域研究已成為當(dāng)前該領(lǐng)域的最新發(fā)展方向。高性能計算集群是一類典型的高性能計算架構(gòu)。開展基于高性能計算集群構(gòu)建高性能地理信息系統(tǒng)的方法研究既具有理論價值,也具有實際應(yīng)用價值。其中,高性能集群環(huán)境下的大規(guī)模地理空間數(shù)據(jù)組織與訪問方法是一個重要的研究內(nèi)容,主要包括地理空間數(shù)據(jù)模型、存儲方式、讀寫方法和查詢檢索等內(nèi)容,目標是實現(xiàn)集群環(huán)境下大規(guī)模地理空間數(shù)據(jù)的統(tǒng)一組織管理和高效訪問。本文圍繞上述目標,研究了高性能計算集群下的地理空間數(shù)據(jù)模型、柵格數(shù)據(jù)并行訪問方法、矢量數(shù)據(jù)推薦檢索等內(nèi)容,工作包含以下四個部分:(1)地理空間資源統(tǒng)一組織模型的構(gòu)建針對高性能計算集群環(huán)境下的地理空間資源種類繁多的現(xiàn)狀,采用了組合模式,對地理空間資源進行了抽象建模、統(tǒng)一描述,屏蔽了不同類型數(shù)據(jù)的個性特征,為后續(xù)的算法設(shè)計打下了基礎(chǔ),實現(xiàn)了對大規(guī)模地理空間資源的有效組織。(2)地理柵格數(shù)據(jù)的并行IO方法針對提高大規(guī)模柵格數(shù)據(jù)IO效率的需求,研究了集群環(huán)境下地理柵格數(shù)據(jù)的并行IO模式,提出了一種地理柵格數(shù)據(jù)并行IO方法:通過采用基于元數(shù)據(jù)的分割方法,為每個進程設(shè)置獨立的寫數(shù)據(jù)視圖,對大量的零散寫數(shù)據(jù)進程進行了聚合,改善了大規(guī)模柵格數(shù)據(jù)讀寫性能,縮短了大規(guī)模柵格數(shù)據(jù)IO時間。(3)空間數(shù)據(jù)的地名自動標注與檢索針對空間數(shù)據(jù)的數(shù)據(jù)規(guī)模變大后檢索準確率降低的情況,本文提出了基于最小外包框的空間數(shù)據(jù)自動標注算法和基于凸包的矢量數(shù)據(jù)自動標注算法。通過對空間數(shù)據(jù)地名信息的提取和標注,算法為大規(guī)�？臻g數(shù)據(jù)的高效檢索提供了方便,為提高空間數(shù)據(jù)檢索的準確率,改善訪問性能作出了有益的嘗試。(4)地理空間數(shù)據(jù)管理原型系統(tǒng)的實現(xiàn)在前文的基礎(chǔ)上,論文設(shè)計實現(xiàn)了一個高性能集群下的地理空間資源管理原型系統(tǒng),對論文所提出的各種方法進行了測試驗證。
[Abstract]:With the arrival of big data era, the geospatial data in the field of geographic information system (GIS) presents more and more large data volume, higher data precision and greater computational complexity. The traditional geographic information system based on single computer has been difficult to solve the complex geographic computing problem which is data intensive and computationally intensive. At the same time, with the rapid development of multi-core, multi-core GPU, cluster and other new computing hardware architecture, the performance is constantly improving and the cost is declining, which provides an effective way for the design and implementation of high-performance GIS platform based on these architectures. Compared with the traditional GIS, high performance GIS brings many advantages, such as the expansion of data processing scale, the improvement of data integration, the convenience of function coordination, and so on, which is beneficial to expand the scope of application. To promote the development of large-scale geographic information applications. Therefore, the research of high performance GIS based on new hardware architecture has become the latest development direction in this field. High performance computing cluster is a typical high performance computing architecture. The method of constructing high performance geographic information system based on high performance computing cluster has both theoretical value and practical application value. Among them, large-scale geospatial data organization and access method in high performance cluster environment is an important research content, including geospatial data model, storage mode, read and write method and query and retrieval, etc. The goal is to realize the unified organization, management and efficient access of large scale geospatial data in cluster environment. In this paper, the geo-spatial data model, the parallel access method of raster data, the vector data recommendation retrieval and so on under the high performance computing cluster are studied. The work includes the following four parts: (1) the construction of the unified organization model of geospatial resources, aiming at the present situation of various kinds of geospatial resources in the high performance computing cluster environment, adopts the combination mode, and carries on the abstract modeling to the geospatial resources. The unified description masked the personality characteristics of different types of data and laid the foundation for the subsequent algorithm design. The efficient organization of large-scale geospatial resources is realized. (2) in order to improve the efficiency of large scale spatial data IO, the parallel IO mode of geographic grid data in cluster environment is studied. This paper presents a parallel IO method for geographic grid data. By using the metadata segmentation method, we set up an independent write data view for each process, and aggregate a large number of scattered write data processes. The performance of large scale raster data reading and writing is improved, and the time of large scale raster data IO is shortened. (3) when the scale of spatial data becomes larger, the retrieval accuracy rate decreases. In this paper, a spatial data automatic tagging algorithm based on minimum outsourced frame and a vector data automatic tagging algorithm based on convex hull are proposed. By extracting and annotating the toponymic information of spatial data, the algorithm provides convenience for efficient retrieval of large-scale spatial data, and improves the accuracy of spatial data retrieval. A beneficial attempt has been made to improve access performance. (4) based on the previous implementation of the Geospatial data Management prototype system, a geo-spatial resource management prototype system based on high performance cluster is designed and implemented in this paper. The methods proposed in this paper are tested and verified.
【學(xué)位授予單位】：國防科學(xué)技術(shù)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：P208
，
本文編號：2152717
[Abstract]:With the arrival of big data era, the geospatial data in the field of geographic information system (GIS) presents more and more large data volume, higher data precision and greater computational complexity. The traditional geographic information system based on single computer has been difficult to solve the complex geographic computing problem which is data intensive and computationally intensive. At the same time, with the rapid development of multi-core, multi-core GPU, cluster and other new computing hardware architecture, the performance is constantly improving and the cost is declining, which provides an effective way for the design and implementation of high-performance GIS platform based on these architectures. Compared with the traditional GIS, high performance GIS brings many advantages, such as the expansion of data processing scale, the improvement of data integration, the convenience of function coordination, and so on, which is beneficial to expand the scope of application. To promote the development of large-scale geographic information applications. Therefore, the research of high performance GIS based on new hardware architecture has become the latest development direction in this field. High performance computing cluster is a typical high performance computing architecture. The method of constructing high performance geographic information system based on high performance computing cluster has both theoretical value and practical application value. Among them, large-scale geospatial data organization and access method in high performance cluster environment is an important research content, including geospatial data model, storage mode, read and write method and query and retrieval, etc. The goal is to realize the unified organization, management and efficient access of large scale geospatial data in cluster environment. In this paper, the geo-spatial data model, the parallel access method of raster data, the vector data recommendation retrieval and so on under the high performance computing cluster are studied. The work includes the following four parts: (1) the construction of the unified organization model of geospatial resources, aiming at the present situation of various kinds of geospatial resources in the high performance computing cluster environment, adopts the combination mode, and carries on the abstract modeling to the geospatial resources. The unified description masked the personality characteristics of different types of data and laid the foundation for the subsequent algorithm design. The efficient organization of large-scale geospatial resources is realized. (2) in order to improve the efficiency of large scale spatial data IO, the parallel IO mode of geographic grid data in cluster environment is studied. This paper presents a parallel IO method for geographic grid data. By using the metadata segmentation method, we set up an independent write data view for each process, and aggregate a large number of scattered write data processes. The performance of large scale raster data reading and writing is improved, and the time of large scale raster data IO is shortened. (3) when the scale of spatial data becomes larger, the retrieval accuracy rate decreases. In this paper, a spatial data automatic tagging algorithm based on minimum outsourced frame and a vector data automatic tagging algorithm based on convex hull are proposed. By extracting and annotating the toponymic information of spatial data, the algorithm provides convenience for efficient retrieval of large-scale spatial data, and improves the accuracy of spatial data retrieval. A beneficial attempt has been made to improve access performance. (4) based on the previous implementation of the Geospatial data Management prototype system, a geo-spatial resource management prototype system based on high performance cluster is designed and implemented in this paper. The methods proposed in this paper are tested and verified.
【學(xué)位授予單位】：國防科學(xué)技術(shù)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：P208
，
本文編號：2152717
本文鏈接：http://sikaile.net/kejilunwen/dizhicehuilunwen/2152717.html
最近更新
教材專著
