本文選題：Hadoop + 并行解壓��； 參考：《河北師范大學(xué)》2015年碩士論文

【摘要】：近年來國(guó)家對(duì)地震領(lǐng)域的發(fā)展頗為關(guān)注,地震數(shù)據(jù)分析成為地震領(lǐng)域的熱門話題。隨著科學(xué)技術(shù)的不斷發(fā)展,地震數(shù)據(jù)采集儀器的精密度有了很大提升。分散在國(guó)內(nèi)的地震監(jiān)測(cè)臺(tái)站數(shù)量不斷增加,這使得數(shù)據(jù)采集量大增。國(guó)家臺(tái)網(wǎng)中心每天匯集約40GB的地震波形數(shù)據(jù),如此海量的數(shù)據(jù)給數(shù)據(jù)存儲(chǔ)和分析造成了很大的挑戰(zhàn)。為了方便傳輸,通常將波形數(shù)據(jù)保存為SEED格式。在進(jìn)行地震數(shù)據(jù)分析前,需要首先將SEED格式文件解壓縮,得到原始樣本序列。現(xiàn)有方法是利用串行思想,每次只能夠解壓縮單個(gè)文件,無(wú)法完成批量文件的解壓縮操作。由于地震數(shù)據(jù)在采集過程中會(huì)收集到一些干擾信號(hào),這些信號(hào)會(huì)影響地震分析的準(zhǔn)確性,因此在解壓縮得到原始樣本序列后,需要先對(duì)序列進(jìn)行濾波處理,保證地震數(shù)據(jù)分析的質(zhì)量。面對(duì)大量地震波形數(shù)據(jù)時(shí),串行濾波方法存在計(jì)算速度慢,處理效率較低的問題。Hadoop是開源的分布式計(jì)算框架,也是目前使用較廣泛的云計(jì)算技術(shù)之一。其核心組件分布式文件系統(tǒng)可以實(shí)現(xiàn)海量數(shù)據(jù)的可靠存儲(chǔ),Map Reduce編程模型可以通過并行化方式處理大規(guī)模數(shù)據(jù),縮短數(shù)據(jù)處理時(shí)間,因此可以利用Hadoop平臺(tái)來解決上述問題。本文從如下方面進(jìn)行了研究:(1)基于地震波形數(shù)據(jù)文件的格式特征,對(duì)在Hadoop平臺(tái)上并行處理該類數(shù)據(jù)文件進(jìn)行可行性分析。(2)利用Map Reduce并行編程模型思想,設(shè)計(jì)了與地震波形數(shù)據(jù)文件相匹配的輸入格式,提出并實(shí)現(xiàn)了批量地震波形數(shù)據(jù)文件的并行解壓縮算法。其中采用二次排序方法確定通道內(nèi)數(shù)據(jù)記錄的排列順序,保證解壓后數(shù)據(jù)拼接的正確性。(3)針對(duì)解壓后得到的數(shù)據(jù),通過分析濾波操作原理,結(jié)合Map Reduce并行編程模型特點(diǎn),提出了一種并行濾波處理算法,實(shí)現(xiàn)了多通道數(shù)據(jù)的并行濾波操作。(4)搭建小型Hadoop集群環(huán)境,對(duì)提出的兩種算法進(jìn)行實(shí)驗(yàn)測(cè)試,分析實(shí)驗(yàn)結(jié)果及算法性能。
[Abstract]:In recent years, the country pays close attention to the development of the seismic field, and seismic data analysis has become a hot topic in the seismic field. With the development of science and technology, the precision of seismic data acquisition instrument has been greatly improved. The number of seismic monitoring stations scattered in China is increasing, which makes the data acquisition increase greatly. The National Network Center collects about 40 GB of seismic waveform data every day, which poses a great challenge to data storage and analysis. In order to facilitate transmission, waveform data is usually saved as seed format. Before seismic data analysis, the seed format file should be decompressed to obtain the original sample sequence. The existing method is to decompress a single file at a time by using the idea of serial, and can not complete the decompression operation of batch file. Because some interference signals will be collected in the process of seismic data acquisition, these signals will affect the accuracy of seismic analysis. Therefore, after decompressing and obtaining the original sample sequence, we need to filter the sequence first. Ensure the quality of seismic data analysis. In the face of a large number of seismic waveform data, the serial filtering method has the problem of slow computation speed and low processing efficiency. Hadoop is an open source distributed computing framework and one of the most widely used cloud computing technologies. Its core component, distributed file system, can realize the reliable storage map reduce programming model of massive data. It can process large scale data by parallelization and shorten data processing time. Therefore, Hadoop platform can be used to solve the above problems. In this paper, the following aspects are studied: (1) based on the format features of seismic waveform data file, the feasibility of parallel processing of this kind of data file on Hadoop platform is analyzed. (2) the idea of parallel programming model based on Map reduce is used to analyze the feasibility of parallel processing of this kind of data file on Hadoop platform. The input format matching the seismic waveform data file is designed and the parallel decompression algorithm of batch seismic waveform data file is proposed and implemented. Among them, the secondary sorting method is used to determine the arrangement order of the data records in the channel to ensure the correctness of the data splicing after decompression. (3) according to the decompressed data, the filtering operation principle is analyzed and the characteristics of the Map reduce parallel programming model are combined. A parallel filtering algorithm is proposed to realize the parallel filtering operation of multi-channel data. (4) A small Hadoop cluster environment is built and the two algorithms are tested and the experimental results and algorithm performance are analyzed.
【學(xué)位授予單位】：河北師范大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2015
【分類號(hào)】：P315.63;TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 張軍華;呂寧;田連玉;陸文志;鐘磊;;地震資料去噪方法技術(shù)綜合評(píng)述[J];地球物理學(xué)進(jìn)展;2006年02期

2 王洪體,陳陽(yáng),莊燦濤;SEED格式STEIM2數(shù)據(jù)壓縮算法在實(shí)時(shí)地震數(shù)據(jù)傳輸中的應(yīng)用[J];地震地磁觀測(cè)與研究;2004年04期

3 張e，

本文編號(hào)：2067371