本文選題：測(cè)試模型 + 數(shù)據(jù)分片��； 參考：《華東理工大學(xué)》2013年碩士論文

【摘要】：互聯(lián)網(wǎng)技術(shù)的發(fā)展和廣泛應(yīng)用導(dǎo)致產(chǎn)生了大量的數(shù)據(jù),關(guān)系型數(shù)據(jù)庫(kù)無(wú)法滿足新興業(yè)務(wù)的需求。為了應(yīng)對(duì)海量數(shù)據(jù)的存儲(chǔ)和管理需求,各大廠商或組織紛紛推出了不同的NoSQL存儲(chǔ)方案,HBase就是其中著名的方案之一,在各個(gè)行業(yè)有著非常廣泛的應(yīng)用。在此背景下本文開展了HBase存儲(chǔ)方案性能測(cè)試方法的研究。 首先,本文介紹了NoSQL數(shù)據(jù)庫(kù)的興起和發(fā)展過(guò)程,回顧了數(shù)據(jù)庫(kù)性能測(cè)試的研究現(xiàn)狀和性能測(cè)試的一般過(guò)程,發(fā)現(xiàn)傳統(tǒng)的性能測(cè)試方法沒有充分的考慮系統(tǒng)架構(gòu)對(duì)性能的影響,而現(xiàn)行的NoSQL數(shù)據(jù)庫(kù)系統(tǒng)架構(gòu)中的一些新特性,如系統(tǒng)的動(dòng)態(tài)調(diào)度、數(shù)據(jù)的多副本存放策略和可配置性對(duì)性能都有著極其重要的影響。同時(shí)在特定業(yè)務(wù)中的不同用戶行為下,系統(tǒng)的性能也存在著很大的差異。基于此,本文提出了NoSQL數(shù)據(jù)庫(kù)系統(tǒng)性能測(cè)試的模型,涵蓋架構(gòu)驅(qū)動(dòng)的性能測(cè)試和業(yè)務(wù)驅(qū)動(dòng)的性能測(cè)試兩個(gè)方面,彌補(bǔ)了傳統(tǒng)方法的不足,并以HBase為例開展了實(shí)踐研究。 基于架構(gòu)驅(qū)動(dòng)的性能測(cè)試,本文研究了HBase面向列族存儲(chǔ)的數(shù)據(jù)模型、數(shù)據(jù)的讀寫過(guò)程、數(shù)據(jù)表的分片式存儲(chǔ)和數(shù)據(jù)的備份存儲(chǔ)策略等方面的具體實(shí)現(xiàn)方式,分析了其中影響其性能的架構(gòu)要素,包括同等數(shù)據(jù)大小下同一列族的單域與多域、單列族和多列族、數(shù)據(jù)分片的大小以及數(shù)據(jù)備份因子等,基于這些架構(gòu)中的性能影響因素分別設(shè)計(jì)并實(shí)現(xiàn)了相應(yīng)的測(cè)試方案,測(cè)試了各要素對(duì)HBase性能的影響程度。 基于業(yè)務(wù)驅(qū)動(dòng)的性能測(cè)試,本文研究了HBase在搜索引擎業(yè)務(wù)中的性能測(cè)試方法。根據(jù)搜索引擎業(yè)務(wù)中用戶訪問(wèn)量的變化具有周期性這一特點(diǎn),本文提出了基于時(shí)間序列的潛周期測(cè)試模型,并將該模型應(yīng)用到搜索引擎業(yè)務(wù)驅(qū)動(dòng)的HBase性能測(cè)試上,另外在建模的過(guò)程中使用小波技術(shù)對(duì)異常用戶訪問(wèn)量數(shù)據(jù)進(jìn)行了過(guò)濾。同時(shí)為了更加真實(shí)的反映HBase在搜索引擎業(yè)務(wù)中的性能,設(shè)計(jì)了基于用戶行為特征的性能測(cè)試方案,并以YCSB測(cè)試套件為基礎(chǔ)具體實(shí)現(xiàn)并驗(yàn)證了這種測(cè)試方案。 最后,對(duì)本文的工作進(jìn)行了總結(jié),并對(duì)將來(lái)的工作進(jìn)行了展望。
[Abstract]:With the development and wide application of Internet technology, large amounts of data are produced, and relational databases can not meet the needs of new business. In order to meet the demand of storage and management of massive data, various manufacturers or organizations have introduced different NoSQL storage schemes, which is one of the famous ones, and has been widely used in various industries. In this context, the performance testing method of HBase storage scheme is studied. First of all, this paper introduces the rise and development of NoSQL database, reviews the research status of database performance testing and the general process of performance testing, and finds that the traditional performance testing methods do not fully consider the impact of system architecture on performance. However, some new features in the current NoSQL database architecture, such as dynamic scheduling of the system, multi-replica storage strategy and configurability of data, have an extremely important impact on performance. At the same time, the performance of the system also varies greatly under the different user behavior in the specific business. Based on this, this paper puts forward a performance test model of NoSQL database system, which covers two aspects: architecture driven performance test and business driven performance test, which make up for the shortcomings of traditional methods, and take HBase as an example to carry out practical research. Based on the performance test driven by architecture, this paper studies the implementation of HBase data model for column family storage, the process of data reading and writing, the piecewise storage of data tables and the backup storage strategy of data, etc. The architectural elements that affect its performance are analyzed, including single-domain and multi-domain of the same column family under the same data size, single-column family and multi-column family, size of data slice and data backup factor, etc. Based on the performance factors in these architectures, the corresponding test schemes are designed and implemented, and the influence of each factor on the performance of HBase is tested. Based on business-driven performance testing, this paper studies the performance testing method of HBase in search engine business. According to the periodicity of users' visits in search engine business, a time series based latent cycle test model is proposed in this paper, and the model is applied to search engine business-driven HBase performance testing. In addition, wavelet technology is used to filter the abnormal user access data in the process of modeling. At the same time, in order to reflect the performance of HBase in search engine business more realistically, a performance test scheme based on user behavior characteristics is designed, and the test scheme is implemented and verified based on YCSB test suite. Finally, the work of this paper is summarized and the future work is prospected.
【學(xué)位授予單位】：華東理工大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前7條

1 佘青;;利用Apache Jmeter進(jìn)行Web性能測(cè)試的研究[J];智能計(jì)算機(jī)與應(yīng)用;2012年02期

2 閆宏飛,李曉明;關(guān)于中國(guó)Web的大小、形狀和結(jié)構(gòu)[J];計(jì)算機(jī)研究與發(fā)展;2002年08期

3 韓光濤,張?jiān)?SPECweb99:最新Web服務(wù)器性能基準(zhǔn)測(cè)試[J];計(jì)算機(jī)應(yīng)用研究;2000年04期

4 李昆霖;;淺析性能測(cè)試[J];科技信息;2012年09期

5 桑圣洪;胡飛;;性能測(cè)試工具LoadRunner的工作機(jī)理及關(guān)鍵技術(shù)研究[J];科學(xué)技術(shù)與工程;2007年06期

6 于戈,王國(guó)仁,王欣暉,鄭懷遠(yuǎn);一個(gè)面向?qū)ο髷?shù)據(jù)庫(kù)系統(tǒng)的TPC-C測(cè)試與分析[J];軟件學(xué)報(bào);1999年09期

7 白敬培;潘清;馮建峰;王映東;;分布式海量數(shù)據(jù)管理系統(tǒng)Hypertable關(guān)鍵技術(shù)分析[J];網(wǎng)絡(luò)安全技術(shù)與應(yīng)用;2009年05期

相關(guān)碩士學(xué)位論文 前2條

1 劉亮;基于HTTP/S協(xié)議的Web性能測(cè)試工具的設(shè)計(jì)與實(shí)現(xiàn)[D];內(nèi)蒙古大學(xué);2005年

2 劉洋;復(fù)雜業(yè)務(wù)環(huán)境下軟件性能測(cè)試模型的設(shè)計(jì)與應(yīng)用[D];東北大學(xué)　;2009年

，

本文編號(hào)：1880661