本文選題：Bigtable + 批量插入��； 參考：《西安電子科技大學(xué)》2014年碩士論文

【摘要】：云存儲(chǔ)是云計(jì)算的核心內(nèi)容，其市場(chǎng)潛力巨大。Bigtable分布式數(shù)據(jù)存儲(chǔ)系統(tǒng)是Google為其內(nèi)部海量的結(jié)構(gòu)化數(shù)據(jù)開(kāi)發(fā)的云存儲(chǔ)技術(shù)，是Google的第三項(xiàng)云計(jì)算關(guān)鍵技術(shù)。Bigtable具有高可靠性、高性能、可伸縮等特性，是所有云時(shí)代分布式存儲(chǔ)系統(tǒng)的開(kāi)發(fā)藍(lán)本。然而，Bigtable中仍存在許多亟待改進(jìn)的地方。 本文首先對(duì)Bigtable的實(shí)現(xiàn)細(xì)節(jié)進(jìn)行具體的表述，并在深入了解Bigtable架構(gòu)、逐步剖析Bigtable實(shí)現(xiàn)細(xì)節(jié)的基礎(chǔ)上，提出了Bigtable在批量插入、主服務(wù)器故障恢復(fù)、SSTable壓縮機(jī)制、子表服務(wù)器重啟、時(shí)間軸訪問(wèn)效率低和讀操作延遲方面的問(wèn)題。并在系統(tǒng)分析各個(gè)問(wèn)題的產(chǎn)生原因之后，給出了這些問(wèn)題可能的解決方案。對(duì)于每種方案，都通過(guò)仿真或性能分析，給出了新方案在解決Bigtable原有問(wèn)題上的優(yōu)勢(shì)。 本文針對(duì)Bigtable常規(guī)方法在批量插入時(shí)會(huì)導(dǎo)致子表頻繁分裂的問(wèn)題，給出了帶預(yù)估期的批量插入方案。對(duì)于Bigtable主服務(wù)器重啟操作流程復(fù)雜、資源和時(shí)間消耗多的問(wèn)題，本文將檢查點(diǎn)機(jī)制應(yīng)用到主服務(wù)器的恢復(fù)中，給出了帶檢查點(diǎn)的主服務(wù)器恢復(fù)機(jī)制。對(duì)于Bigtable原壓縮機(jī)制對(duì)周期的選取要求較高的問(wèn)題，給出了一種基于數(shù)量的壓縮方案。對(duì)于Bigtable中子表服務(wù)器重啟不能有效利用其本地磁盤(pán)上的SSTable文件，，增加不必要的網(wǎng)絡(luò)通信和數(shù)據(jù)傳輸?shù)膯?wèn)題，本文給出了子表服務(wù)器初始化的新方案。對(duì)于Bigtable在時(shí)間軸上訪問(wèn)低效的問(wèn)題，本文給出了帶時(shí)間索引的SSTable設(shè)計(jì)。對(duì)于Bigtable中元操作延遲的問(wèn)題，本文給出了讓子表服務(wù)器一邊從GFS上讀取數(shù)據(jù)，一邊與客戶端通訊將數(shù)據(jù)返回給客戶端的方案。
[Abstract]:Cloud storage is the core of cloud computing. Its market potential is huge. BigTable distributed data storage system is a cloud storage technology developed by Google for its internal mass of structured data. It is the third key technology of Google cloud computing. BigTable has high reliability. High performance, scalability and other features are all cloud era distributed storage system development blueprint. However, there are still many areas for improvement in BigTable. In this paper, the implementation details of Bigtable are described in detail, and on the basis of deeply understanding the Bigtable architecture and analyzing the implementation details of Bigtable step by step, this paper puts forward the Bigtable in batch insertion, the main server fault recovery and the compression mechanism of SSTable, and the reboot of the sub-table server. Time axis access inefficiency and read delay problems. After systematically analyzing the causes of each problem, the possible solutions to these problems are given. For each scheme, the advantages of the new scheme in solving the original problem of Bigtable are given through simulation or performance analysis. In this paper, a batch insertion scheme with prediction period is proposed to solve the problem that Bigtable routine method causes frequent splitting of subtables during batch insertion. For the problems of complex operation flow and more resource and time consumption of the Bigtable master server restart, this paper applies the checkpoint mechanism to the recovery of the primary server, and gives the recovery mechanism of the primary server with checkpointing. This paper presents a quantity-based compression scheme for the problem that the Bigtable compressor system requires a higher selection of the cycle. To solve the problem that the Bigtable neutron table server can not effectively utilize the SSTable file on its local disk and add unnecessary network communication and data transmission, a new scheme for initialization of the sub-table server is presented in this paper. For the problem of Bigtable accessing inefficiency on time axis, this paper gives the design of SSTable with time index. For the problem of meta operation delay in Bigtable, this paper presents a scheme to let the subtable server read data from the GFS while communicating with the client to return the data to the client.
【學(xué)位授予單位】：西安電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2014
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前2條

1 蔡鍵;王樹(shù)梅;;基于Google的云計(jì)算實(shí)例分析[J];電腦知識(shí)與技術(shù);2009年25期

2 張榆;馬友忠;孟小峰;;一種基于HBase的高效空間關(guān)鍵字查詢策略[J];小型微型計(jì)算機(jī)系統(tǒng);2012年10期

本文編號(hào)：1880485