【摘要】：以高分辨率遙感衛(wèi)星為代表的新一代數(shù)據(jù)獲取技術(shù)取得了較大進(jìn)步,地理柵格數(shù)據(jù)在時空分辨率、數(shù)據(jù)類型、覆蓋面積等方面不斷提高,為地理應(yīng)用提供了更多的數(shù)據(jù)信息,然而這些提高也帶來數(shù)據(jù)量幾何級增長,但同時也致使傳統(tǒng)的遙感數(shù)據(jù)處理方法無法滿足大規(guī)模地理柵格數(shù)據(jù)計算與分析需求。因此,研究面向大規(guī)模地理柵格數(shù)據(jù)的高性能計算方法與體系,進(jìn)而為提高開發(fā)效率和解決復(fù)雜地學(xué)問題的能力具有十分重要的意義。 為了解決大規(guī)模柵格地理數(shù)據(jù)并行處理所面臨的難題,本文系統(tǒng)研究高性能計算架構(gòu)下大規(guī)模地理柵格數(shù)據(jù)并行處理方法,引入MPI (Message Passing Interface)和MP (Multi processing)作為基礎(chǔ)并行環(huán)境,研究柵格數(shù)據(jù)處理算法程序必備的功能和必須的流程,綜合所有算法的共性,采用設(shè)計模式思想,構(gòu)造符合面向?qū)ο蟪绦蚧驹瓌t的系統(tǒng)框架體系,構(gòu)建一種協(xié)作式大規(guī)模柵格數(shù)據(jù)的并行處理框架(Cooperative Big Geographic Raster Data Parallel Processing Framework, CBGRDPPF),并結(jié)合地理柵格數(shù)據(jù)并行類型與復(fù)雜度等處理任務(wù)特點,探討了在此框架下進(jìn)行地學(xué)柵格數(shù)據(jù)處理任務(wù)的協(xié)同處理方法和技術(shù),分析不同參數(shù)和環(huán)境對其運行速度、并行效率的影響,實驗驗證局域并行效率,實現(xiàn)對并行處理框架的優(yōu)化,從而建立一種解決復(fù)雜地學(xué)問題的并行開發(fā)協(xié)作模式,為地理柵格數(shù)據(jù)的高效處理提供一種新的解決方法和技術(shù)支撐。本文的主要研究成果體現(xiàn)在如下幾個方面： (1)提出了一種大規(guī)模地理柵格數(shù)據(jù)并行處理算法的并行解耦方法 以MP工和MP作為基礎(chǔ)并行環(huán)境,將地理柵格計算部分和并行計算支撐部分分別進(jìn)行抽象和封裝,作為相互協(xié)作的部件予以松耦合地裝配和執(zhí)行,有效地分離并行計算體系與地學(xué)問題的強(qiáng)耦合。 (2)構(gòu)建了協(xié)作式大規(guī)模柵格數(shù)據(jù)的并行處理框架 在地理柵格數(shù)據(jù)并行處理機(jī)制及并行解耦方法研究的基礎(chǔ)上,建立了適合于協(xié)同開發(fā)的數(shù)據(jù)塊分塊、分發(fā)、縫合的數(shù)據(jù)類模型,為實現(xiàn)地學(xué)柵格數(shù)據(jù)協(xié)同并行奠定了基礎(chǔ)；建立了核心算法的封裝類模型及開發(fā)策略,實現(xiàn)了代碼開發(fā)與算法細(xì)節(jié)分離,保證了并行計算與分析應(yīng)用工作的協(xié)同。 (3)提出了基于并行解耦思想的地理柵格數(shù)據(jù)全局計算并行化方法 針對內(nèi)存一次無法裝載的大型柵格地理數(shù)據(jù)卻要進(jìn)行地理柵格數(shù)據(jù)的全局計算,分析地學(xué)處理算法原理,基于CBGRDPPF框架利用對數(shù)據(jù)的橫向、縱向劃分、過程數(shù)據(jù)的分塊暫存等策略,使各個并行進(jìn)程在占用有限的內(nèi)存空間情況下,分塊依次處理整個柵格數(shù)據(jù),大大降低了開發(fā)算法程序的復(fù)雜度,從而實現(xiàn)復(fù)雜并行地理數(shù)據(jù)計算任務(wù)高效并行。 (4)提出了基于并行解耦思想的地理柵格數(shù)據(jù)動態(tài)計算并行化方法 動態(tài)計算主要是指一些柵格數(shù)據(jù)的聚類算法步驟未知,計算過程動態(tài)迭代。本文通過FCM算法為代表的聚類算法,基于CBGRDPPF框架,通過數(shù)據(jù)多策略的劃分、序列化讀取、計算同步和廣播機(jī)制,實現(xiàn)一種動態(tài)計算的地理柵格數(shù)據(jù)并行處理方法,從而解決了不平衡計算量的并行化問題。
[Abstract]:The new generation of data acquisition technology, represented by high resolution remote sensing satellite, has made great progress. Geographic grid data has been improved in time and space resolution, data type, covering area and so on. It provides more data information for geographic applications. However, these improvements have also brought the geometric growth of data, but it also leads to traditional data. The remote sensing data processing method can not meet the needs of large-scale geographic grid data calculation and analysis. Therefore, it is of great significance to study the high performance computing methods and systems for large-scale geographic grid data, and to improve the efficiency of development and the ability to solve complex geological problems.
In order to solve the problem of large scale grid geographic data parallel processing, this paper systematically studies the parallel processing method of large-scale geographic grid data under high performance computing architecture, introduces MPI (Message Passing Interface) and MP (Multi processing) as the basic parallel environment, and studies the necessary functions of the grid data processing algorithm program. The necessary process, the generality of all algorithms, the idea of design pattern, a system framework that conforms to the basic principles of object-oriented programs, and a parallel processing framework for collaborative large-scale raster data (Cooperative Big Geographic Raster Data Parallel Processing Framework, CBGRDPPF), combined with geographic grid. With the characteristics of data parallel type and complexity, the cooperative processing method and technology of geoscience grid data processing task under this framework are discussed, and the influence of different parameters and environment on its running speed and parallel efficiency is analyzed. The experiment verifies the local parallel efficiency and the optimization of parallel processing framework. The parallel development and collaboration model for solving complex Geosciences problems provides a new solution and technical support for the efficient processing of geographic grid data. The main research results of this paper are reflected in the following aspects:
(1) a parallel decoupling method for large scale grid data parallel processing algorithm is proposed.
Using MP and MP as the basic parallel environment, the geographic grid computing part and the parallel computing support part are abstracted and encapsulated respectively. As a cooperative component, it is assembled and executed loosely coupled and effectively separates the strong coupling of the parallel computing system and the geoscience problem.
(2) build a parallel processing framework for collaborative large-scale raster data.
On the basis of the parallel processing mechanism of geographic grid data and the research of parallel decoupling method, a data class model which is suitable for collaborative development of data block, distribution and suture is established, which lays the foundation for the realization of geoscience grid data coordination and parallel. The package model and development strategy of the core algorithm are established, and the code development and calculation are realized. Separation of the method details ensures the synergy of parallel computation and analysis application.
(3) a parallel computing method for global raster data based on parallel decoupling is proposed.
In view of the large grid geographic data that can not be loaded once, the global calculation of geographic grid data should be carried out, the principle of the analysis of geoscience processing algorithm is analyzed. Based on the CBGRDPPF framework, the strategies of lateral, longitudinal division of the data and the partitioning of process data are used to make the parallel processes block under the limited memory space. Processing the whole raster data in a row, greatly reduces the complexity of the development algorithm program, and realizes efficient parallel computing of complex parallel geographic data.
(4) a parallel computing method for dynamic computation of geographic raster data based on parallel decoupling is proposed.
The dynamic calculation mainly refers to the unknown clustering algorithm of some grid data and the dynamic iteration of the calculation process. In this paper, the clustering algorithm, represented by the FCM algorithm, is based on the CBGRDPPF framework, through the division of multiple strategies, serialization reading and computing synchronization and broadcasting mechanism, and a dynamic computation of the parallel processing of geographic grid data is realized. The problem of parallelization of unbalanced computation is solved.
【學(xué)位授予單位】：首都師范大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2014
【分類號】：P208

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 陳輝;孫雷鳴;李錄明;羅省賢;趙安軍;;基于MPI+OpenMP的多層次并行偏移算法研究[J];成都理工大學(xué)學(xué)報(自然科學(xué)版);2010年05期

2 秦其明;遙感圖像自動解譯面臨的問題與解決的途徑[J];測繪科學(xué);2000年02期

3 盧麗君,廖明生,張路;分布式并行計算技術(shù)在遙感數(shù)據(jù)處理中的應(yīng)用[J];測繪信息與工程;2005年03期

4 方金云,何建邦;并行柵格數(shù)據(jù)處理網(wǎng)格服務(wù)節(jié)點軟件的關(guān)鍵技術(shù)[J];地球信息科學(xué);2004年01期

5 廖順寶,孫九林,李澤輝,馬琳,彭梅;地學(xué)數(shù)據(jù)產(chǎn)品的開發(fā)、發(fā)布與共享[J];地球科學(xué)進(jìn)展;2005年02期

6 靳華中,孟令奎,王顯;集群環(huán)境下并行GIS的體系結(jié)構(gòu)設(shè)計[J];地理空間信息;2005年05期

7 高建勇;;柵格數(shù)據(jù)結(jié)構(gòu)研究綜述[J];經(jīng)營管理者;2009年18期

8 唐天兵;謝祥宏;申文杰;韋凌云;嚴(yán)毅;;多核CPU環(huán)境下的并行遺傳算法的研究[J];廣西大學(xué)學(xué)報(自然科學(xué)版);2009年04期

9 周玉科;馬廷;周成虎;高錫章;范俊甫;;MySQL集群與MPI的并行空間分析系統(tǒng)設(shè)計與實驗[J];地球信息科學(xué)學(xué)報;2012年04期

10 楊海平;沈占鋒;駱劍承;吳煒;;海量遙感數(shù)據(jù)的高性能地學(xué)計算應(yīng)用與發(fā)展分析[J];地球信息科學(xué)學(xué)報;2013年01期

相關(guān)博士學(xué)位論文 前2條

1 張男;基于內(nèi)容的光學(xué)遙感圖像檢索關(guān)鍵技術(shù)研究[D];國防科學(xué)技術(shù)大學(xué);2008年

2 羅耀華;高性能計算在高光譜遙感數(shù)據(jù)處理中的應(yīng)用研究[D];成都理工大學(xué);2013年

相關(guān)碩士學(xué)位論文 前1條

1 李平;基于FPGA的矩陣特征值并行計算研究[D];重慶大學(xué);2013年

，

本文編號：2122605