本文關(guān)鍵詞： 高性能計(jì)算 海量遙感數(shù)據(jù) 集群計(jì)算 計(jì)算存儲(chǔ)一體化 分布式存儲(chǔ)　出處：《河南大學(xué)》2014年碩士論文　論文類型：學(xué)位論文

【摘要】：高性能計(jì)算機(jī)的迅速發(fā)展為海量遙感數(shù)據(jù)處理提供了強(qiáng)大的計(jì)算資源，基于集群及計(jì)算存儲(chǔ)一體化的高性能計(jì)算已成為目前最主要的計(jì)算平臺(tái)。在集群環(huán)境下設(shè)計(jì)并實(shí)現(xiàn)海量遙感數(shù)據(jù)并行處理系統(tǒng)是提高遙感數(shù)據(jù)處理速度的必然選擇。 隨著高分辨率及傳感器類型的增加，遙感數(shù)據(jù)TB級(jí)的增長(zhǎng)對(duì)高性能遙感數(shù)據(jù)處理平臺(tái)的I/O要求造成非常大的壓力，包括服務(wù)器磁盤的讀寫壓力，服務(wù)器網(wǎng)段的傳輸壓力等，為了最大化發(fā)揮分布式并行優(yōu)勢(shì)，在保證系統(tǒng)安全穩(wěn)定的前提下，本文采用了計(jì)算存儲(chǔ)一體化策略，以減少并行處理系統(tǒng)中的數(shù)據(jù)傳輸。 本文在分析了網(wǎng)格、云計(jì)算及集群計(jì)算等幾種高性能計(jì)算的優(yōu)缺點(diǎn)基礎(chǔ)上，提出并實(shí)現(xiàn)了一種基于計(jì)算存儲(chǔ)一體化的高性能集群架構(gòu)，以實(shí)現(xiàn)對(duì)海量遙感數(shù)據(jù)的高效處理，并在衛(wèi)星遙感基礎(chǔ)共性產(chǎn)品一體化處理系統(tǒng)中得以應(yīng)用。 本文的工作主要體現(xiàn)在以下幾點(diǎn)： （1）基于已有集群系統(tǒng)，深入研究遙感數(shù)據(jù)高性能計(jì)算處理系統(tǒng)，，針對(duì)海量遙感數(shù)據(jù)對(duì)處理平臺(tái)I/O、服務(wù)器磁盤的讀寫及網(wǎng)絡(luò)傳輸造成的壓力等問題，采用把數(shù)據(jù)存儲(chǔ)從原來集中到一個(gè)機(jī)器變成計(jì)算、存儲(chǔ)一體化架構(gòu)，最大限度降低系統(tǒng)架構(gòu)的I/O傳輸，更充分利用了存儲(chǔ)服務(wù)器的計(jì)算資源、計(jì)算服務(wù)器的存儲(chǔ)資源，從而提高平臺(tái)的整理處理時(shí)效。 （2）采用了邏輯上靜態(tài)、物理上可動(dòng)態(tài)擴(kuò)展的分布式存儲(chǔ)模型來實(shí)現(xiàn)對(duì)海量遙感數(shù)據(jù)的存儲(chǔ)及管理。引入虛擬磁盤空間（VDS）的概念來把切片分配到多個(gè)存儲(chǔ)站點(diǎn)，并采用交叉?zhèn)浞莘绞酱_保系統(tǒng)的穩(wěn)定性。 （3）采用三級(jí)并行策略來最大化發(fā)揮高性能并行計(jì)算在大規(guī)模的遙感影像處理中發(fā)揮著的重要作用 （4）實(shí)驗(yàn)驗(yàn)證。通過使用不同類型衛(wèi)星的數(shù)據(jù)測(cè)試，從多個(gè)角度進(jìn)行對(duì)比驗(yàn)證，證明本文提出的高性能集群架構(gòu)并行處理能力和效率上均有提升。
[Abstract]:The rapid development of high-performance computers provides powerful computing resources for massive remote sensing data processing. The high performance computing based on the integration of cluster and storage has become the most important computing platform. It is an inevitable choice to design and implement the parallel processing system of massive remote sensing data in cluster environment to improve the speed of remote sensing data processing. With the increase of high resolution and sensor type, the increase of TB level of remote sensing data causes great pressure on the I / O requirement of high performance remote sensing data processing platform, including the pressure of reading and writing of server disk, the pressure of transmission of server network segment, etc. In order to maximize the advantage of distributed parallelism, under the premise of ensuring the security and stability of the system, this paper adopts the strategy of integration of computing and storage to reduce the data transmission in parallel processing system. On the basis of analyzing the advantages and disadvantages of grid, cloud computing and cluster computing, this paper proposes and implements a kind of high performance cluster architecture based on computing and storage integration, which can efficiently process the massive remote sensing data. And it is applied in the integrated processing system of satellite remote sensing basic commonality product. The work of this paper is mainly reflected in the following points:. (1) based on the existing cluster system, the high performance computing and processing system of remote sensing data is studied in depth. Aiming at the problems caused by the massive remote sensing data on the processing platform I / O, the read, write and transmission of server disk, and so on, By changing the data storage from the original centralized to a single machine into a computing, storage integration framework, the I / O transmission of the system architecture is minimized, and the computing resources of the storage server and the storage resources of the computing server are more fully utilized. In order to improve the platform finishing and processing time. The distributed storage model, which is logically static and dynamically scalable in physics, is used to store and manage the massive remote sensing data. The concept of virtual disk space (VDS) is introduced to distribute slices to multiple storage sites. Cross-backup is used to ensure the stability of the system. Using the three-level parallel strategy to maximize the high performance parallel computing plays an important role in large-scale remote sensing image processing. By using the data of different types of satellites, the parallel processing capability and efficiency of the proposed high performance cluster architecture are proved to be improved.
【學(xué)位授予單位】：河南大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2014
【分類號(hào)】：TP79

【參考文獻(xiàn)】

相關(guān)期刊論文 前9條

1 陳康;;云計(jì)算后臺(tái)大規(guī)模數(shù)據(jù)處理技術(shù)探討[J];電信工程技術(shù)與標(biāo)準(zhǔn)化;2009年11期

2 李圣強(qiáng);李閩峰;劉桂平;王斌;吳婷;王浩;;高性能集群計(jì)算系統(tǒng)的構(gòu)建[J];地震;2012年01期

3 呂雪鋒;程承旗;龔健雅;關(guān)麗;;海量遙感數(shù)據(jù)存儲(chǔ)管理技術(shù)綜述[J];中國(guó)科學(xué):技術(shù)科學(xué);2011年12期

4 劉偉;劉露;陳犖;鐘志農(nóng);;海量遙感影像數(shù)據(jù)存儲(chǔ)技術(shù)研究[J];計(jì)算機(jī)工程;2009年05期

5 楊任農(nóng);白娟;黃震宇;鄔蒙;樊蓉;;基于SQLite的LOD模式海量影像數(shù)據(jù)管理系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];計(jì)算機(jī)工程與科學(xué);2011年10期

6 郭建平;肖華東;劉昭華;曹春香;張顥;光潔;;基于并行計(jì)算的氣溶膠定量遙感反演模型實(shí)現(xiàn)[J];計(jì)算機(jī)應(yīng)用;2009年06期

7 賴積保;羅曉麗;余濤;賈培艷;;一種支持云計(jì)算的遙感影像數(shù)據(jù)組織模型研究[J];計(jì)算機(jī)科學(xué);2013年07期

8 陳康;鄭緯民;;云計(jì)算:系統(tǒng)實(shí)例與研究現(xiàn)狀[J];軟件學(xué)報(bào);2009年05期

9 王小偉,郭力,葛蔚,楊章遠(yuǎn);高性能并行集群計(jì)算環(huán)境的構(gòu)建與性能測(cè)試[J];小型微型計(jì)算機(jī)系統(tǒng);2004年03期

相關(guān)博士學(xué)位論文 前1條

1 康俊鋒;云計(jì)算環(huán)境下高分辨率遙感影像存儲(chǔ)與高效管理技術(shù)研究[D];浙江大學(xué);2011年

本文編號(hào)：1489905