本文選題：遙感數(shù)據(jù) + 分布式文件系統(tǒng)�。� 參考：《中國(guó)科學(xué)院大學(xué)(工程管理與信息技術(shù)學(xué)院)》2013年碩士論文

【摘要】：隨著全球?qū)Φ赜^測(cè)技術(shù)的快速發(fā)展,遙感影像數(shù)據(jù)的規(guī)模成指數(shù)倍數(shù)的增長(zhǎng)。同期我國(guó)開展了一系列基礎(chǔ)專項(xiàng)和科研項(xiàng)目,如高分辨率對(duì)地觀測(cè)系統(tǒng)等。這些項(xiàng)目的發(fā)展產(chǎn)生了大量的高分辨率遙感影像數(shù)據(jù),傳統(tǒng)的遙感數(shù)據(jù)存儲(chǔ)管理技術(shù)面對(duì)TB級(jí)至PB級(jí)的數(shù)據(jù)存儲(chǔ)問題越來越吃力。由此引發(fā)了人們對(duì)一系列的超大規(guī)模海量遙感數(shù)據(jù)存儲(chǔ)問題的關(guān)注和研究。如何能夠快速而高效的對(duì)海量遙感數(shù)據(jù)進(jìn)行存取管理是未來幾年內(nèi)人們關(guān)注和研究的一個(gè)重要課題。 本文針對(duì)如何能快速而高效的進(jìn)行海量遙感影像數(shù)據(jù)存儲(chǔ)管理的技術(shù),進(jìn)行了深入研究。選取了hadoop的分布式文件系統(tǒng)HDFS作為存儲(chǔ)平臺(tái),對(duì)比了其他主流的遙感影像數(shù)據(jù)存儲(chǔ)方案,在HDFS文件系統(tǒng)的基礎(chǔ)上,針對(duì)遙感影像數(shù)據(jù),引入了一些其他的優(yōu)良機(jī)制,使之可以應(yīng)用于海量遙感數(shù)據(jù)存儲(chǔ)上。主要的研究?jī)?nèi)容包括： (a)對(duì)傳統(tǒng)的遙感影像數(shù)據(jù)存儲(chǔ)技術(shù)進(jìn)行了分析,探討了常用的傳統(tǒng)遙感影像數(shù)據(jù)存儲(chǔ)在面對(duì)迅猛發(fā)展的數(shù)據(jù)規(guī)模和數(shù)據(jù)多樣性中存在的不足,對(duì)比了現(xiàn)階段主流的分布式文件系統(tǒng)之后,選用了HDFS進(jìn)行遙感數(shù)據(jù)存儲(chǔ)技術(shù)研究。 (b)介紹了傳統(tǒng)的遙感影像數(shù)據(jù)存儲(chǔ)方法—影像四叉樹技術(shù),傳統(tǒng)的四叉樹算法需要消耗大量計(jì)算資源,實(shí)時(shí)性和效率很難保證。因此,本文基于分布式文件系統(tǒng)的核心理念MapReduce算法,提出了四叉樹快速構(gòu)建算法,利用網(wǎng)格節(jié)點(diǎn)的計(jì)算資源快速構(gòu)建四叉樹。并提出了HDFS文件系統(tǒng)下的四叉樹構(gòu)建方式和構(gòu)建策略。 (c)設(shè)計(jì)了基于Hbase數(shù)據(jù)庫(kù)的遙感空間數(shù)據(jù)存儲(chǔ)模型,使之能夠應(yīng)用于HDFS分布式文件系統(tǒng)當(dāng)中；針對(duì)HDFS只有單個(gè)元數(shù)據(jù)節(jié)點(diǎn)NameNode這種情況,所可能存在的系統(tǒng)穩(wěn)定性問題,借鑒了目前主流應(yīng)用系統(tǒng)的機(jī)制,采用雙機(jī)熱備的方式來保證系統(tǒng)的容錯(cuò)性；引入了Nagios管理插件,監(jiān)控分布式文件系統(tǒng)中網(wǎng)格節(jié)點(diǎn)的性能信息,從而保證系統(tǒng)的穩(wěn)定性。 (d)為了解決海量數(shù)據(jù)的高效率服務(wù)問題,在參考了OGC的標(biāo)準(zhǔn)后,本文基于HDFS文件系統(tǒng)中設(shè)計(jì)了一套數(shù)據(jù)服務(wù)接口,并能夠及時(shí)的反饋系統(tǒng)中的數(shù)據(jù)信息和系統(tǒng)狀態(tài)信息。 (e)基于上述的研究思路設(shè)計(jì)了實(shí)驗(yàn),從而驗(yàn)證了本文改進(jìn)策略和方法是有效的。 研究結(jié)果表明了本文采用了基于HDFS分布式文件系統(tǒng)對(duì)遙感影像數(shù)據(jù)進(jìn)行集中管理,針對(duì)HDFS而設(shè)計(jì)的高性能四叉樹構(gòu)建算法和數(shù)據(jù)存儲(chǔ)模型,可以解決日益增長(zhǎng)的超大規(guī)模海量遙感數(shù)據(jù)存儲(chǔ)管理的問題。同時(shí)針對(duì)HDFS在存儲(chǔ)管理數(shù)據(jù)過程中存在的問題進(jìn)行的優(yōu)化和改進(jìn),能夠表現(xiàn)出比原有的系統(tǒng)更優(yōu)的性能,因此這些優(yōu)化和改進(jìn)是有效的。
[Abstract]:With the rapid development of global Earth observation technology, the scale of remote sensing image data increases exponentially. In the same period, China has carried out a series of basic projects and scientific research projects, such as high resolution Earth observation system. The development of these projects has resulted in a large number of high-resolution remote sensing image data, the traditional remote sensing data storage management technology facing TB to PB level of data storage problem is becoming more and more difficult. This has aroused the attention and research of a series of massive remote sensing data storage problems. How to access and manage the massive remote sensing data quickly and efficiently is an important issue that people pay attention to and study in the coming years. In this paper, we study how to store and manage massive remote sensing image data quickly and efficiently. The distributed file system of hadoop is selected as storage platform, and other mainstream remote sensing image data storage schemes are compared. On the basis of HDFS file system, some other excellent mechanisms are introduced for remote sensing image data. So that it can be applied to mass remote sensing data storage. The main research contents are as follows: (a) analyzes the traditional remote sensing image data storage technology, and discusses the shortcomings of the traditional remote sensing image data storage in the face of the rapid development of data scale and data diversity. After comparing the mainstream distributed file system at present, we choose HDFS to store remote sensing data;. (b) introduces the traditional remote sensing image data storage method-image quadtree technology. The traditional quadtree algorithm needs to consume a lot of computing resources, so it is difficult to guarantee the real-time and efficiency. Therefore, based on the core idea of distributed file system, MapReduce algorithm, this paper proposes a fast quadtree construction algorithm, which uses the computing resources of grid nodes to quickly construct quadtree. This paper also puts forward the construction method and strategy of quadtree in HDFS file system. The remote sensing spatial data storage model based on Hbase database is designed by. (c), which can be applied to HDFS distributed file system. In view of the fact that HDFS has only a single metadata node, NameNode, the system stability problem that may exist in HDFS is discussed. The mechanism of current mainstream application systems is used to ensure the fault tolerance of the system, and the Nagios management plug-in is introduced. To monitor the performance information of grid nodes in distributed file system, to ensure the stability of the system,. (d) has referred to the standard of. (d) in order to solve the high efficiency service problem of massive data. This paper designs a set of data service interface based on HDFS file system, and can timely feedback the data information and system state information in the system. (e). Based on the above research ideas, the experiment is designed. It is proved that the improved strategy and method are effective. The results show that the distributed file system based on HDFS is used for centralized management of remote sensing image data, and the high performance quadtree algorithm and data storage model are designed for HDFS. It can solve the problem of storage and management of large and large scale remote sensing data. At the same time, the optimization and improvement of HDFS in the process of storage and management data can show better performance than the original system, so these optimization and improvement are effective.
【學(xué)位授予單位】：中國(guó)科學(xué)院大學(xué)(工程管理與信息技術(shù)學(xué)院)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333;TP751

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 方裕,周成虎,景貴飛,陸鋒,駱劍承;第四代GIS軟件研究[J];中國(guó)圖象圖形學(xué)報(bào);2001年09期

，

本文編號(hào)：2087539