空間科學(xué)大數(shù)據(jù)存儲模型SP-HDF及應(yīng)用研究
本文選題:空間天氣科學(xué)數(shù)據(jù) 切入點(diǎn):存儲模型 出處:《中國地質(zhì)大學(xué)(北京)》2013年博士論文 論文類型:學(xué)位論文
【摘要】:空間科學(xué)是研究發(fā)生在日地空間、行星際空間及至整個宇宙空間的物理、天文、化學(xué)及生命等自然現(xiàn)象及其規(guī)律的科學(xué)?臻g科學(xué)觀測和研究過程中將產(chǎn)生數(shù)量巨大、種類繁多、數(shù)據(jù)結(jié)構(gòu)復(fù)雜、計(jì)算密集型的科學(xué)數(shù)據(jù),有效地存儲、管理、共享這些科學(xué)數(shù)據(jù)已經(jīng)成為空間科學(xué)應(yīng)用研究迫切需要解決的一個重要問題。本文充分分析了海量空間科學(xué)數(shù)據(jù)的特征,深入研究了HDF(Hierarchy Data Format)、CDF(Common DataForm)等新的存儲技術(shù)。在此基礎(chǔ)上,,提出一種新的空間科學(xué)大數(shù)據(jù)自描述存儲模型SP-HDF(Space-HDF)?紤]到目前標(biāo)準(zhǔn)化和非標(biāo)準(zhǔn)化數(shù)據(jù)并存的情況,作者提出了一種基于元數(shù)據(jù)描述的虛擬觀測站點(diǎn)共享服務(wù)模型。本文的研究成果及創(chuàng)新點(diǎn)包括: (1)提出一種針對空間科學(xué)大數(shù)據(jù)存儲管理和集成共享的自描述存儲模型SP-HDF。針對空間天氣和子午工程空間科學(xué)數(shù)據(jù)的特點(diǎn)和邏輯結(jié)構(gòu),設(shè)計(jì)了相應(yīng)的邏輯存儲結(jié)構(gòu),在此基礎(chǔ)上,為用戶設(shè)計(jì)了簡單、易用、友好的數(shù)據(jù)訪問接口。SP-HDF存儲模型的優(yōu)勢在于屏蔽了用戶直接訪問底層HDF和CDF文件的細(xì)節(jié),因而降低了數(shù)據(jù)訪問的復(fù)雜性。另一方面邏輯存儲結(jié)構(gòu)直接面向領(lǐng)域應(yīng)用,提升了數(shù)據(jù)訪問粒度和數(shù)據(jù)抽象級別,在實(shí)際應(yīng)用中更加易于理解。 (2)針對目前大量空間科學(xué)數(shù)據(jù)在短期內(nèi)無法完全標(biāo)準(zhǔn)化的現(xiàn)狀,提出一種基于虛擬觀測站點(diǎn)的共享服務(wù)模型。該模型通過共享服務(wù)元數(shù)據(jù)模型和抽象數(shù)據(jù)服務(wù)接口,實(shí)現(xiàn)多元、異構(gòu)科學(xué)數(shù)據(jù)的集成和共享服務(wù)。 (3)設(shè)計(jì)并研發(fā)了一個基于SP-HDF存儲模型的空間天氣科學(xué)數(shù)據(jù)邏輯結(jié)構(gòu)及各種大數(shù)據(jù)服務(wù)接口的原型系統(tǒng)。通過對核心數(shù)據(jù)接口的訪問,原型系統(tǒng)實(shí)現(xiàn)了與數(shù)值模擬、可視化等多個系統(tǒng)的集成應(yīng)用。驗(yàn)證了基于虛擬觀測站點(diǎn)的共享服務(wù)模型的子午工程數(shù)據(jù)服務(wù)接口原型,實(shí)現(xiàn)了數(shù)據(jù)檢索、數(shù)據(jù)下載、數(shù)據(jù)可視化等多個功能。 (4)設(shè)計(jì)并實(shí)現(xiàn)了基于XML半形式化描述和數(shù)據(jù)處理模板技術(shù)的子午工程科學(xué)數(shù)據(jù)標(biāo)準(zhǔn)化處理模型,解決了多元、異構(gòu)、海量空間科學(xué)數(shù)據(jù)的高效處理和靈活定制的困難問題。
[Abstract]:Space science is research in physics, sun earth space, interplanetary space and the whole space astronomy, chemistry and life and other natural phenomena and laws of science. Space science research and observation in the process of large quantity, variety, complex data structure, scientific data intensive computing, efficient storage, management. The scientific data sharing has become an important issue in the research of space science applications need to be solved urgently. This paper analyses the characteristics of massive space science data, in-depth study of the HDF (Hierarchy Data Format), CDF (Common DataForm) and other new storage technology. On this basis, put forward a new kind of space science data self description storage model of SP-HDF (Space-HDF). Considering the current standardized and non standardized data coexist, the author proposed a method according to the virtual observation station based on quaternion description The research results and innovation points of this paper are as follows:
(1) is proposed for space science data storage management and self characteristics and logical structure description storage model of SP-HDF. for space weather and meridian project space science data integration and sharing, design the logical storage structure, on this basis, for the user to design a simple, easy to use, friendly interface of data access.SP-HDF the storage model has the advantage of shielding users direct access to the underlying HDF and CDF files in detail, thus reducing the complexity of data access. On the other hand the logical storage structure directly facing the field of application, enhance the level of abstraction and granularity of data access, more easy to understand in the practical application.
(2) according to the current situation of large space science data in the short term can not completely standardized, proposed a shared service model for virtual stations. Based on the model to realize multiple through shared service metadata model and abstract data service interface, service, integration and sharing of heterogeneous scientific data.
(3) the design and development of a SP-HDF storage model based on space weather science data logical structure and all kinds of large data service interface prototype system. Based on the core data access interface, prototype system has realized the numerical simulation and visualization, integration of multiple systems should be used. To verify the data service interface prototype of meridian project shared service model of virtual observation stations based on the implementation of data retrieval, data download, data visualization and other functions.
(4) a standardized processing model of meridian engineering data based on XML semi formal description and data processing template technology is designed and implemented, which solves the difficult problems of high efficiency and flexible customization of multivariate, heterogeneous and massive spatial scientific data.
【學(xué)位授予單位】:中國地質(zhì)大學(xué)(北京)
【學(xué)位級別】:博士
【學(xué)位授予年份】:2013
【分類號】:TP333
【參考文獻(xiàn)】
相關(guān)期刊論文 前10條
1 劉文軍;李靖;袁昌洪;劉方;解令運(yùn);;基于NetCDF數(shù)據(jù)模型的氣象資料存儲設(shè)計(jì)[J];安徽農(nóng)業(yè)科學(xué);2009年03期
2 王玲,龔健雅;基于HDF文件的組織方式與影像提取[J];測繪通報(bào);2003年04期
3 黃耀裔;陳文成;;時態(tài)GIS的應(yīng)用——以福建省GDP演變?yōu)槔齕J];測繪與空間地理信息;2009年03期
4 劉潤達(dá);諸云強(qiáng);;科學(xué)數(shù)據(jù)共享關(guān)鍵問題探索——以地球系統(tǒng)科學(xué)數(shù)據(jù)共享網(wǎng)為例[J];地理科學(xué)進(jìn)展;2007年05期
5 石勇,魏奉思,馮學(xué)尚,葉占銀;1997年1月事件行星際擾動在三維背景太陽風(fēng)中傳播的數(shù)值模擬[J];地球物理學(xué)報(bào);2001年03期
6 趙蘇璇;羅堅(jiān);楊成蔭;;基于BP神經(jīng)網(wǎng)絡(luò)的氣象格點(diǎn)數(shù)據(jù)無損壓縮方法[J];地球科學(xué)進(jìn)展;2008年02期
7 王永韜,劉良明;HDF5格式特點(diǎn)及其對遙感數(shù)據(jù)格式標(biāo)準(zhǔn)化的幾點(diǎn)啟示[J];國土資源遙感;2005年03期
8 王永壽;陳延輝;;電磁流體力學(xué)技術(shù)在航空航天領(lǐng)域的應(yīng)用[J];飛航導(dǎo)彈;2009年03期
9 梅清;李浩;;CORBA在遠(yuǎn)程訪問HDF5文件系統(tǒng)中的應(yīng)用[J];湖北工業(yè)大學(xué)學(xué)報(bào);2006年04期
10 李安虎,周玉斌,劉海行,盧銘;基于WebGIS的海洋科學(xué)數(shù)據(jù)共享平臺的分析與設(shè)計(jì)[J];海洋科學(xué)進(jìn)展;2004年01期
本文編號:1584776
本文鏈接:http://sikaile.net/kejilunwen/jisuanjikexuelunwen/1584776.html