基于NoSQL技術(shù)的MUSER數(shù)據(jù)歸檔與發(fā)布系統(tǒng)的設(shè)計與實現(xiàn)
發(fā)布時間：2018-12-18 05:25
【摘要】：隨著高空間、高時間、高頻率分辨率的觀測設(shè)備不斷投入使用,天文觀測產(chǎn)生的數(shù)據(jù)飛速增長。傳統(tǒng)的科學(xué)數(shù)據(jù)歸檔和處理方法遠(yuǎn)遠(yuǎn)不能滿足海量天文數(shù)據(jù)的需求。為了提高現(xiàn)代天文觀測的科學(xué)產(chǎn)出效率,必須探索新的數(shù)據(jù)歸檔與發(fā)布方法。本文以明安圖超寬頻譜射電日像儀(Mingantu Ultrawide SpEctral Radioheliograph, MUSER)項目為研究背景,隨著MUSER項目進(jìn)入觀測階段,如何快速有效地歸檔其多波段多通道觀測產(chǎn)生的海量數(shù)據(jù)成為急需解決的問題。同時,為便于觀測數(shù)據(jù)的共享,在歸檔的基礎(chǔ)上應(yīng)采取有效的方式對觀測信息進(jìn)行發(fā)布。本研究的工作主要包括四個方面：(1)通過基于NoSQL技術(shù)的FastBit技術(shù)和傳統(tǒng)的關(guān)系型數(shù)據(jù)庫MySQL技術(shù)在建立索引、查詢、插入數(shù)據(jù)及歸檔空間大小上的對比實驗,得出FastBit更適合MUSER的實際索引和檢索需求。(2)利用模擬數(shù)據(jù)設(shè)計并實現(xiàn)了MUSER項目的數(shù)據(jù)歸檔系統(tǒng)原型,包括FITS、UV-FITS格式和PNG格式的數(shù)據(jù)歸檔。FITS文件在歸檔的同時建立索引,便于后期的數(shù)據(jù)檢索。PNG格式的數(shù)據(jù)歸檔應(yīng)用于項目的發(fā)布,基于MUSER按天觀測的特性,歸檔的層級目錄按天定時創(chuàng)建。(3)利用模擬數(shù)據(jù)實現(xiàn)了將MUSER原始數(shù)據(jù)處理生成的太陽圖像與其相關(guān)觀測信息的融合,生成的圖像用于MUSER的發(fā)布系統(tǒng)原型,進(jìn)行效果顯示。(4)設(shè)計并實現(xiàn)了MUSER項目的數(shù)據(jù)發(fā)布系統(tǒng)原型,其中的發(fā)布包括每隔10min的圖像發(fā)布和每日中天時間圖像的動態(tài)發(fā)布。同時,發(fā)布系統(tǒng)實現(xiàn)了指定的時間間隔的動態(tài)效果展示。本文研究和實驗的數(shù)據(jù)歸檔和發(fā)布系統(tǒng)原型,能應(yīng)用在MUSER的實際歸檔與發(fā)布系統(tǒng)中,對其提供技術(shù)支撐。
[Abstract]:With the use of high space, high time and high frequency resolution observation equipment, astronomical observation data is increasing rapidly. Traditional scientific data archiving and processing methods can not meet the needs of massive astronomical data. In order to improve the efficiency of scientific output of modern astronomical observations, a new method of data archiving and publishing must be explored. In this paper, with the background of the (Mingantu Ultrawide SpEctral Radioheliograph, MUSER) project of the ultra-wide spectrum radio heliograph of the Ming Antu, with the entry of the MUSER project into the observation stage, How to quickly and effectively file the massive data generated by multi-band and multi-channel observations has become an urgent problem. At the same time, in order to facilitate the sharing of observation data, we should adopt an effective way to publish the observation information on the basis of archiving. The work of this study mainly includes four aspects: (1) the comparative experiment of establishing index, querying, inserting data and archiving space through FastBit technology based on NoSQL technology and MySQL technology of traditional relational database. It is concluded that FastBit is more suitable for the actual index and retrieval requirements of MUSER. (2) the prototype of MUSER project data archiving system, including FITS, is designed and implemented by using simulated data. Data archiving in UV-FITS format and PNG format. FITS files are indexed at the same time in order to facilitate the later data retrieval. PNG format data archiving is applied to the release of projects. Based on the characteristics of MUSER daily observation, The archived hierarchical catalogue is created on a daily basis. (3) using simulated data, the solar image generated by MUSER raw data processing is fused with its related observation information, and the generated image is used in the prototype of the MUSER publishing system. (4) the prototype of the MUSER project data publishing system is designed and implemented, which includes the release of every 10min image and the dynamic distribution of the daily mid-day image. At the same time, the release system realizes the dynamic effect display of the specified time interval. The prototype of data archiving and publishing system studied and experimented in this paper can be applied to the actual archiving and publishing system of MUSER and provide technical support to it.
【學(xué)位授予單位】：昆明理工大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2015
【分類號】：P111.41;TP311.13
本文編號：2385417
[Abstract]:With the use of high space, high time and high frequency resolution observation equipment, astronomical observation data is increasing rapidly. Traditional scientific data archiving and processing methods can not meet the needs of massive astronomical data. In order to improve the efficiency of scientific output of modern astronomical observations, a new method of data archiving and publishing must be explored. In this paper, with the background of the (Mingantu Ultrawide SpEctral Radioheliograph, MUSER) project of the ultra-wide spectrum radio heliograph of the Ming Antu, with the entry of the MUSER project into the observation stage, How to quickly and effectively file the massive data generated by multi-band and multi-channel observations has become an urgent problem. At the same time, in order to facilitate the sharing of observation data, we should adopt an effective way to publish the observation information on the basis of archiving. The work of this study mainly includes four aspects: (1) the comparative experiment of establishing index, querying, inserting data and archiving space through FastBit technology based on NoSQL technology and MySQL technology of traditional relational database. It is concluded that FastBit is more suitable for the actual index and retrieval requirements of MUSER. (2) the prototype of MUSER project data archiving system, including FITS, is designed and implemented by using simulated data. Data archiving in UV-FITS format and PNG format. FITS files are indexed at the same time in order to facilitate the later data retrieval. PNG format data archiving is applied to the release of projects. Based on the characteristics of MUSER daily observation, The archived hierarchical catalogue is created on a daily basis. (3) using simulated data, the solar image generated by MUSER raw data processing is fused with its related observation information, and the generated image is used in the prototype of the MUSER publishing system. (4) the prototype of the MUSER project data publishing system is designed and implemented, which includes the release of every 10min image and the dynamic distribution of the daily mid-day image. At the same time, the release system realizes the dynamic effect display of the specified time interval. The prototype of data archiving and publishing system studied and experimented in this paper can be applied to the actual archiving and publishing system of MUSER and provide technical support to it.
【學(xué)位授予單位】：昆明理工大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2015
【分類號】：P111.41;TP311.13
【參考文獻(xiàn)】
相關(guān)期刊論文 前6條
1 王娟;石建芳;王曉明;劉俊民;邱宏茂;張波;盧遠(yuǎn)鐳;許進(jìn);;一種數(shù)據(jù)歸檔機(jī)制的實現(xiàn)[J];核電子學(xué)與探測技術(shù);2009年04期
2 李戰(zhàn)懷;王國仁;周傲英;;從數(shù)據(jù)庫視角解讀大數(shù)據(jù)的研究進(jìn)展與趨勢[J];計算機(jī)工程與科學(xué);2013年10期
3 吳盛殷,南仁東;射電望遠(yuǎn)鏡的發(fā)展和前景[J];天文學(xué)進(jìn)展;1998年03期
4 申德榮;于戈;王習(xí)特;聶鐵錚;寇月;;支持大數(shù)據(jù)管理的NoSQL系統(tǒng)研究綜述[J];軟件學(xué)報;2013年08期
5 許向陽;李明勝;;位圖索引的設(shè)計與實現(xiàn)[J];微計算機(jī)應(yīng)用;2006年02期
6 王威;顏毅華;張堅;姬國樞;陳志軍;;CSRH陣列設(shè)計研究及饋源設(shè)計的初步考慮[J];天文研究與技術(shù).國家天文臺臺刊;2006年02期
，本文編號：2385417
本文鏈接：http://sikaile.net/kejilunwen/tianwen/2385417.html
教材專著
