本文關(guān)鍵詞：地震資料統(tǒng)一存取中索引機(jī)制的研究　出處：《東北石油大學(xué)》2013年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 索引 大數(shù)據(jù) 統(tǒng)一存取 查詢處理

【摘要】：地震資料是典型的大規(guī)模靜態(tài)數(shù)據(jù)集,即由海量特定的記錄或數(shù)據(jù)塊組成的數(shù)據(jù)集合,因規(guī)模龐大一般存儲(chǔ)于分布式文件系統(tǒng)中,這些數(shù)據(jù)一旦寫(xiě)入便很少再修改。它有多種不同的存儲(chǔ)格式,應(yīng)用程序若為每一種格式都應(yīng)開(kāi)發(fā)一套接口會(huì)有很多不便,不僅自身的代碼量會(huì)大大增加,其靈活性、可移植性、可擴(kuò)展性都會(huì)受到很大的限制。為解決上述問(wèn)題,人們研究了多種解決方案,其中“多格式海量數(shù)據(jù)統(tǒng)一存取”方案提出為數(shù)據(jù)建立統(tǒng)一的邏輯描述方式,通過(guò)對(duì)邏輯層標(biāo)準(zhǔn)化接口的操作實(shí)現(xiàn)對(duì)物理數(shù)據(jù)的統(tǒng)一訪問(wèn)。這種設(shè)計(jì)在具有良好的靈活性的同時(shí)也由于在物理層和應(yīng)用層間添加了邏輯層,增加了操作步驟,存在著效率降低的弊端。 本文依據(jù)地震資料的特點(diǎn),優(yōu)化了地震數(shù)據(jù)統(tǒng)一存取(Unified access of the seismic data,UASD)模型中邏輯層的索引結(jié)構(gòu),提出了基于UASD的分布式索引框架。同時(shí)提出了基于HDFS的一種的可適用于B-樹(shù)和R-樹(shù)及其變種的層次索引結(jié)構(gòu),并給出了索引結(jié)構(gòu)的并行構(gòu)建方式。研究分析地震資料的統(tǒng)一存取模型,對(duì)模型中邏輯層的索引結(jié)構(gòu)進(jìn)行優(yōu)化,提取出可快速定位的索引公式。提出基于UASD和HDFS的分布式索引架構(gòu),以提高UASD的索引性能。提出基于地震道頭關(guān)鍵字的分級(jí)建索策略,避免非活躍關(guān)鍵字索引占用空間及維護(hù)的開(kāi)銷。研究分析海量數(shù)據(jù)中常用索引機(jī)制,提出基于HDFS的一種可適用于B-樹(shù)和R-樹(shù)及其變種的層次索引結(jié)構(gòu),改變鍵-值存儲(chǔ)結(jié)構(gòu)在非主鍵索引過(guò)程中的劣勢(shì)。提出層次索引的并行建立方法,加快索引構(gòu)建速度。最后綜合本文的研究?jī)?nèi)容,介紹UASD模式在地震資料并行處理平臺(tái)中的應(yīng)用,以此來(lái)驗(yàn)證本文提出的相關(guān)技術(shù)和算法的可行性和有效性。 實(shí)驗(yàn)表明,該系列方法減少了查詢響應(yīng)時(shí)間、索引空間占用和數(shù)據(jù)傳輸開(kāi)銷,有效的提高了UASD的索引性能。
[Abstract]:Seismic data is a typical large-scale static data set, that is, a data set consisting of a large number of specific records or data blocks, because of its large scale, it is generally stored in distributed file systems. Once written, this data is rarely modified. It has many different storage formats, and it would be inconvenient for an application to develop an interface for each format, not just its own code. Its flexibility, portability, scalability will be greatly limited. In order to solve the above problems, people have studied a variety of solutions. One of the "multi-format massive data unified access" scheme proposed for the establishment of a unified logical description of the data. The uniform access to physical data is realized through the operation of the logical layer standardized interface. This design has good flexibility but also adds the logical layer between the physical layer and the application layer and adds the operation steps. There is a drawback of reduced efficiency. According to the characteristics of seismic data, the unified access of the seismic data is optimized. The index structure of the logical layer in the UASD model. A distributed index framework based on UASD and a hierarchical index structure based on HDFS for B- tree, R- tree and its varieties are proposed. The parallel construction method of index structure is given. The unified access model of seismic data is analyzed and the index structure of logic layer in the model is optimized. The index formula which can be located quickly is extracted. The distributed index architecture based on UASD and HDFS is proposed to improve the index performance of UASD. A hierarchical cable construction strategy based on seismic trace head keyword is proposed. To avoid the overhead of using inactive keyword index and maintaining it, the common indexing mechanism in mass data is studied and analyzed. This paper presents a hierarchical index structure based on HDFS, which can be applied to B- tree, R- tree and its varieties, and changes the disadvantage of key-value storage structure in the process of non-primary key index. Finally, the application of UASD model in the parallel processing platform of seismic data is introduced. In order to verify the feasibility and effectiveness of the related techniques and algorithms proposed in this paper. The experiments show that this series of methods reduce query response time, index space consumption and data transfer overhead, and effectively improve the index performance of UASD.
【學(xué)位授予單位】：東北石油大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 張林,文必龍,陳仁華,王東,張忠民;POSC數(shù)據(jù)存取技術(shù)[J];大慶石油學(xué)院學(xué)報(bào);1998年03期

2 吳廣君;王樹(shù)鵬;陳明;李超;;海量結(jié)構(gòu)化數(shù)據(jù)存儲(chǔ)檢索系統(tǒng)[J];計(jì)算機(jī)研究與發(fā)展;2012年S1期

3 吳煒;蘇永紅;李瑞軒;盧正鼎;;基于DHT的分布式索引技術(shù)研究與實(shí)現(xiàn)[J];計(jì)算機(jī)科學(xué);2010年02期

4 牛德姣;蔡濤;詹永照;鞠時(shí)光;;海量存儲(chǔ)系統(tǒng)中的元數(shù)據(jù)分級(jí)索引算法[J];計(jì)算機(jī)應(yīng)用研究;2012年02期

5 趙鳳芝;饒平;劉永江;;多格式海量數(shù)據(jù)統(tǒng)一存取的設(shè)計(jì)與研究[J];科學(xué)技術(shù)與工程;2011年09期

6 馬承杰;;地震數(shù)據(jù)訪問(wèn)技術(shù)研究[J];石油工業(yè)計(jì)算機(jī)應(yīng)用;2005年04期

相關(guān)博士學(xué)位論文 前2條

1 龍柏;并行計(jì)算平臺(tái)上的數(shù)據(jù)索引技術(shù)研究[D];中國(guó)科學(xué)技術(shù)大學(xué);2011年

2 敖富江;數(shù)據(jù)流頻繁模式挖掘關(guān)鍵算法及其仿真應(yīng)用研究[D];國(guó)防科學(xué)技術(shù)大學(xué);2008年

相關(guān)碩士學(xué)位論文 前5條

1 楊岳;非結(jié)構(gòu)化數(shù)據(jù)統(tǒng)一訪問(wèn)平臺(tái)及索引技術(shù)研究[D];解放軍信息工程大學(xué);2010年

2 曹永生;地震數(shù)據(jù)處理及可視化技術(shù)研究與應(yīng)用[D];電子科技大學(xué);2011年

3 饒平;多格式海量數(shù)據(jù)統(tǒng)一存取的效率優(yōu)化[D];東北石油大學(xué);2011年

4 溫慶慶;可視化地震資料解釋系統(tǒng)的研究與開(kāi)發(fā)[D];西安科技大學(xué);2008年

5 賁福才;統(tǒng)一大塊數(shù)據(jù)存取方法的研究[D];大慶石油學(xué)院;2010年

，

本文編號(hào)：1405226