發(fā)布時(shí)間：2018-03-16 03:10

  本文選題：元數(shù)據(jù)　切入點(diǎn)：元數(shù)據(jù)組織　出處：《華中科技大學(xué)》2016年碩士論文　論文類型：學(xué)位論文

【摘要】：隨著大數(shù)據(jù)時(shí)代的到來,面向大數(shù)據(jù)的存儲(chǔ)系統(tǒng)紛紛出現(xiàn)。不斷增長的數(shù)據(jù)量,使得集中式元數(shù)據(jù)管理系統(tǒng)的負(fù)擔(dān)越來越重,逐漸成為大數(shù)據(jù)存儲(chǔ)的瓶頸。為此,人們提出了多種分布式元數(shù)據(jù)管理方法,但由于元數(shù)據(jù)的結(jié)構(gòu)類型復(fù)雜多樣,目前尚沒有一種方法能夠同時(shí)改善元數(shù)據(jù)管理的性能和擴(kuò)展性。提出了一種文件路徑和屬性信息分離的分布式元數(shù)據(jù)組織方法。將元數(shù)據(jù)組織成目錄索引和元數(shù)據(jù)屬性信息兩個(gè)部分,通過構(gòu)建目錄索引,將元數(shù)據(jù)以目錄或小于目錄為單位劃分到不同的桶(Bucket)內(nèi),再根據(jù)元數(shù)據(jù)服務(wù)器集群的負(fù)載情況將桶指派到不同的元數(shù)據(jù)服務(wù)器上。方法利用目錄索引和桶提高元數(shù)據(jù)的管理性能;通過構(gòu)建目錄索引時(shí)考慮集群負(fù)載情況,實(shí)現(xiàn)元數(shù)據(jù)管理的可擴(kuò)展性。此外,提出基于該方法的元數(shù)據(jù)位置緩存策略,策略解決了位置緩存信息不一致的問題,縮短了元數(shù)據(jù)管理的流程。測(cè)試結(jié)果表明,提出的方法能獲得較高的管理性能,特別適合高并發(fā)的情況;具有良好的可擴(kuò)展性和較好的訪問局部性,而且可以不限制目錄的大小;避免了重命名元數(shù)據(jù)造成的不必要的遷移。與集中式元數(shù)據(jù)管理方法對(duì)比,方法采用單一元數(shù)據(jù)服務(wù)器時(shí),元數(shù)據(jù)的創(chuàng)建、查詢等操作性能都有了數(shù)倍的提升。
[Abstract]:With the arrival of big data's era, the storage system for big data appeared one after another. The increasing amount of data makes the burden of centralized metadata management system become more and more heavy, and gradually becomes the bottleneck of big data storage. A variety of distributed metadata management methods have been proposed, but because of the complexity and diversity of the structure of metadata, At present, there is no method to improve the performance and scalability of metadata management simultaneously. A distributed metadata organization method, which separates file path and attribute information, is proposed. The metadata is organized into directory index and metadata. According to two parts of attribute information, By building a directory index, the metadata is divided into different buckets in directories or smaller than directories. Then according to the load of metadata server cluster, the buckets are assigned to different metadata servers. Methods Directory index and bucket are used to improve the management performance of metadata. In addition, a metadata location caching strategy based on this method is proposed, which solves the problem of inconsistent location cache information and shortens the process of metadata management. The test results show that, The proposed method can achieve high management performance, especially suitable for high concurrency, have good scalability and good access locality, and can not limit the size of the directory. Compared with centralized metadata management method, when using single metadata server, the operation performance of metadata creation and query has been improved several times.
【學(xué)位授予單位】：華中科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2016
【分類號(hào)】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前7條

1 肖中正;陳寧江;魏峻;張文博;;一種面向海量存儲(chǔ)系統(tǒng)的高效元數(shù)據(jù)集群管理方案[J];計(jì)算機(jī)研究與發(fā)展;2015年04期

2 羅軍;陳席林;李文生;;高效Key-Value持久化緩存系統(tǒng)的實(shí)現(xiàn)[J];計(jì)算機(jī)工程;2014年03期

3 周江;王偉平;孟丹;馬燦;古曉艷;蔣杰;;面向大數(shù)據(jù)分析的分布式文件系統(tǒng)關(guān)鍵技術(shù)[J];計(jì)算機(jī)研究與發(fā)展;2014年02期

4 徐鵬;陳思;蘇森;;互聯(lián)網(wǎng)應(yīng)用PaaS平臺(tái)體系結(jié)構(gòu)[J];北京郵電大學(xué)學(xué)報(bào);2012年01期

5 韓君易;;NoSQL數(shù)據(jù)庫解決方案Tair淺析[J];電子商務(wù);2011年09期

6 馮幼樂;朱六璋;;CEPH動(dòng)態(tài)元數(shù)據(jù)管理方法分析與改進(jìn)[J];電子技術(shù);2010年09期

7 羅達(dá)強(qiáng);;探析Windows Azure Platform微軟云計(jì)算平臺(tái)[J];硅谷;2010年16期

，

本文編號(hào)：1618057