發(fā)布時間：2018-07-01 10:51

  本文選題：遙感數(shù)據(jù) + 分布式文件系統(tǒng)��； 參考：《中國科學院大學(工程管理與信息技術學院)》2013年碩士論文

【摘要】：隨著全球對地觀測技術的快速發(fā)展,遙感影像數(shù)據(jù)的規(guī)模成指數(shù)倍數(shù)的增長。同期我國開展了一系列基礎專項和科研項目,如高分辨率對地觀測系統(tǒng)等。這些項目的發(fā)展產(chǎn)生了大量的高分辨率遙感影像數(shù)據(jù),傳統(tǒng)的遙感數(shù)據(jù)存儲管理技術面對TB級至PB級的數(shù)據(jù)存儲問題越來越吃力。由此引發(fā)了人們對一系列的超大規(guī)模海量遙感數(shù)據(jù)存儲問題的關注和研究。如何能夠快速而高效的對海量遙感數(shù)據(jù)進行存取管理是未來幾年內(nèi)人們關注和研究的一個重要課題。 本文針對如何能快速而高效的進行海量遙感影像數(shù)據(jù)存儲管理的技術,進行了深入研究。選取了hadoop的分布式文件系統(tǒng)HDFS作為存儲平臺,對比了其他主流的遙感影像數(shù)據(jù)存儲方案,在HDFS文件系統(tǒng)的基礎上,針對遙感影像數(shù)據(jù),引入了一些其他的優(yōu)良機制,使之可以應用于海量遙感數(shù)據(jù)存儲上。主要的研究內(nèi)容包括： (a)對傳統(tǒng)的遙感影像數(shù)據(jù)存儲技術進行了分析,探討了常用的傳統(tǒng)遙感影像數(shù)據(jù)存儲在面對迅猛發(fā)展的數(shù)據(jù)規(guī)模和數(shù)據(jù)多樣性中存在的不足,對比了現(xiàn)階段主流的分布式文件系統(tǒng)之后,選用了HDFS進行遙感數(shù)據(jù)存儲技術研究。 (b)介紹了傳統(tǒng)的遙感影像數(shù)據(jù)存儲方法—影像四叉樹技術,傳統(tǒng)的四叉樹算法需要消耗大量計算資源,實時性和效率很難保證。因此,本文基于分布式文件系統(tǒng)的核心理念MapReduce算法,提出了四叉樹快速構建算法,利用網(wǎng)格節(jié)點的計算資源快速構建四叉樹。并提出了HDFS文件系統(tǒng)下的四叉樹構建方式和構建策略。 (c)設計了基于Hbase數(shù)據(jù)庫的遙感空間數(shù)據(jù)存儲模型,使之能夠應用于HDFS分布式文件系統(tǒng)當中；針對HDFS只有單個元數(shù)據(jù)節(jié)點NameNode這種情況,所可能存在的系統(tǒng)穩(wěn)定性問題,借鑒了目前主流應用系統(tǒng)的機制,采用雙機熱備的方式來保證系統(tǒng)的容錯性；引入了Nagios管理插件,監(jiān)控分布式文件系統(tǒng)中網(wǎng)格節(jié)點的性能信息,從而保證系統(tǒng)的穩(wěn)定性。 (d)為了解決海量數(shù)據(jù)的高效率服務問題,在參考了OGC的標準后,本文基于HDFS文件系統(tǒng)中設計了一套數(shù)據(jù)服務接口,并能夠及時的反饋系統(tǒng)中的數(shù)據(jù)信息和系統(tǒng)狀態(tài)信息。 (e)基于上述的研究思路設計了實驗,從而驗證了本文改進策略和方法是有效的。 研究結果表明了本文采用了基于HDFS分布式文件系統(tǒng)對遙感影像數(shù)據(jù)進行集中管理,針對HDFS而設計的高性能四叉樹構建算法和數(shù)據(jù)存儲模型,可以解決日益增長的超大規(guī)模海量遙感數(shù)據(jù)存儲管理的問題。同時針對HDFS在存儲管理數(shù)據(jù)過程中存在的問題進行的優(yōu)化和改進,能夠表現(xiàn)出比原有的系統(tǒng)更優(yōu)的性能,因此這些優(yōu)化和改進是有效的。
[Abstract]:With the rapid development of global Earth observation technology, the scale of remote sensing image data increases exponentially. In the same period, China has carried out a series of basic projects and scientific research projects, such as high resolution Earth observation system. The development of these projects has resulted in a large number of high-resolution remote sensing image data, the traditional remote sensing data storage management technology facing TB to PB level of data storage problem is becoming more and more difficult. This has aroused the attention and research of a series of massive remote sensing data storage problems. How to access and manage the massive remote sensing data quickly and efficiently is an important issue that people pay attention to and study in the coming years. In this paper, we study how to store and manage massive remote sensing image data quickly and efficiently. The distributed file system of hadoop is selected as storage platform, and other mainstream remote sensing image data storage schemes are compared. On the basis of HDFS file system, some other excellent mechanisms are introduced for remote sensing image data. So that it can be applied to mass remote sensing data storage. The main research contents are as follows: (a) analyzes the traditional remote sensing image data storage technology, and discusses the shortcomings of the traditional remote sensing image data storage in the face of the rapid development of data scale and data diversity. After comparing the mainstream distributed file system at present, we choose HDFS to store remote sensing data;. (b) introduces the traditional remote sensing image data storage method-image quadtree technology. The traditional quadtree algorithm needs to consume a lot of computing resources, so it is difficult to guarantee the real-time and efficiency. Therefore, based on the core idea of distributed file system, MapReduce algorithm, this paper proposes a fast quadtree construction algorithm, which uses the computing resources of grid nodes to quickly construct quadtree. This paper also puts forward the construction method and strategy of quadtree in HDFS file system. The remote sensing spatial data storage model based on Hbase database is designed by. (c), which can be applied to HDFS distributed file system. In view of the fact that HDFS has only a single metadata node, NameNode, the system stability problem that may exist in HDFS is discussed. The mechanism of current mainstream application systems is used to ensure the fault tolerance of the system, and the Nagios management plug-in is introduced. To monitor the performance information of grid nodes in distributed file system, to ensure the stability of the system,. (d) has referred to the standard of. (d) in order to solve the high efficiency service problem of massive data. This paper designs a set of data service interface based on HDFS file system, and can timely feedback the data information and system state information in the system. (e). Based on the above research ideas, the experiment is designed. It is proved that the improved strategy and method are effective. The results show that the distributed file system based on HDFS is used for centralized management of remote sensing image data, and the high performance quadtree algorithm and data storage model are designed for HDFS. It can solve the problem of storage and management of large and large scale remote sensing data. At the same time, the optimization and improvement of HDFS in the process of storage and management data can show better performance than the original system, so these optimization and improvement are effective.
【學位授予單位】：中國科學院大學(工程管理與信息技術學院)
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333;TP751

【參考文獻】

相關期刊論文 前1條

1 方裕,周成虎,景貴飛,陸鋒,駱劍承;第四代GIS軟件研究[J];中國圖象圖形學報;2001年09期

，

本文編號：2087539