發(fā)布時間：2018-05-12 23:34

  本文選題：測試模型 + 數(shù)據(jù)分片��； 參考：《華東理工大學(xué)》2013年碩士論文

【摘要】：互聯(lián)網(wǎng)技術(shù)的發(fā)展和廣泛應(yīng)用導(dǎo)致產(chǎn)生了大量的數(shù)據(jù),關(guān)系型數(shù)據(jù)庫無法滿足新興業(yè)務(wù)的需求。為了應(yīng)對海量數(shù)據(jù)的存儲和管理需求,各大廠商或組織紛紛推出了不同的NoSQL存儲方案,HBase就是其中著名的方案之一,在各個行業(yè)有著非常廣泛的應(yīng)用。在此背景下本文開展了HBase存儲方案性能測試方法的研究。 首先,本文介紹了NoSQL數(shù)據(jù)庫的興起和發(fā)展過程,回顧了數(shù)據(jù)庫性能測試的研究現(xiàn)狀和性能測試的一般過程,發(fā)現(xiàn)傳統(tǒng)的性能測試方法沒有充分的考慮系統(tǒng)架構(gòu)對性能的影響,而現(xiàn)行的NoSQL數(shù)據(jù)庫系統(tǒng)架構(gòu)中的一些新特性,如系統(tǒng)的動態(tài)調(diào)度、數(shù)據(jù)的多副本存放策略和可配置性對性能都有著極其重要的影響。同時在特定業(yè)務(wù)中的不同用戶行為下,系統(tǒng)的性能也存在著很大的差異�；诖�,本文提出了NoSQL數(shù)據(jù)庫系統(tǒng)性能測試的模型,涵蓋架構(gòu)驅(qū)動的性能測試和業(yè)務(wù)驅(qū)動的性能測試兩個方面,彌補了傳統(tǒng)方法的不足,并以HBase為例開展了實踐研究。 基于架構(gòu)驅(qū)動的性能測試,本文研究了HBase面向列族存儲的數(shù)據(jù)模型、數(shù)據(jù)的讀寫過程、數(shù)據(jù)表的分片式存儲和數(shù)據(jù)的備份存儲策略等方面的具體實現(xiàn)方式,分析了其中影響其性能的架構(gòu)要素,包括同等數(shù)據(jù)大小下同一列族的單域與多域、單列族和多列族、數(shù)據(jù)分片的大小以及數(shù)據(jù)備份因子等,基于這些架構(gòu)中的性能影響因素分別設(shè)計并實現(xiàn)了相應(yīng)的測試方案,測試了各要素對HBase性能的影響程度。 基于業(yè)務(wù)驅(qū)動的性能測試,本文研究了HBase在搜索引擎業(yè)務(wù)中的性能測試方法。根據(jù)搜索引擎業(yè)務(wù)中用戶訪問量的變化具有周期性這一特點,本文提出了基于時間序列的潛周期測試模型,并將該模型應(yīng)用到搜索引擎業(yè)務(wù)驅(qū)動的HBase性能測試上,另外在建模的過程中使用小波技術(shù)對異常用戶訪問量數(shù)據(jù)進(jìn)行了過濾。同時為了更加真實的反映HBase在搜索引擎業(yè)務(wù)中的性能,設(shè)計了基于用戶行為特征的性能測試方案,并以YCSB測試套件為基礎(chǔ)具體實現(xiàn)并驗證了這種測試方案。 最后,對本文的工作進(jìn)行了總結(jié),并對將來的工作進(jìn)行了展望。
[Abstract]:With the development and wide application of Internet technology, large amounts of data are produced, and relational databases can not meet the needs of new business. In order to meet the demand of storage and management of massive data, various manufacturers or organizations have introduced different NoSQL storage schemes, which is one of the famous ones, and has been widely used in various industries. In this context, the performance testing method of HBase storage scheme is studied. First of all, this paper introduces the rise and development of NoSQL database, reviews the research status of database performance testing and the general process of performance testing, and finds that the traditional performance testing methods do not fully consider the impact of system architecture on performance. However, some new features in the current NoSQL database architecture, such as dynamic scheduling of the system, multi-replica storage strategy and configurability of data, have an extremely important impact on performance. At the same time, the performance of the system also varies greatly under the different user behavior in the specific business. Based on this, this paper puts forward a performance test model of NoSQL database system, which covers two aspects: architecture driven performance test and business driven performance test, which make up for the shortcomings of traditional methods, and take HBase as an example to carry out practical research. Based on the performance test driven by architecture, this paper studies the implementation of HBase data model for column family storage, the process of data reading and writing, the piecewise storage of data tables and the backup storage strategy of data, etc. The architectural elements that affect its performance are analyzed, including single-domain and multi-domain of the same column family under the same data size, single-column family and multi-column family, size of data slice and data backup factor, etc. Based on the performance factors in these architectures, the corresponding test schemes are designed and implemented, and the influence of each factor on the performance of HBase is tested. Based on business-driven performance testing, this paper studies the performance testing method of HBase in search engine business. According to the periodicity of users' visits in search engine business, a time series based latent cycle test model is proposed in this paper, and the model is applied to search engine business-driven HBase performance testing. In addition, wavelet technology is used to filter the abnormal user access data in the process of modeling. At the same time, in order to reflect the performance of HBase in search engine business more realistically, a performance test scheme based on user behavior characteristics is designed, and the test scheme is implemented and verified based on YCSB test suite. Finally, the work of this paper is summarized and the future work is prospected.
【學(xué)位授予單位】：華東理工大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前7條

1 佘青;;利用Apache Jmeter進(jìn)行Web性能測試的研究[J];智能計算機與應(yīng)用;2012年02期

2 閆宏飛,李曉明;關(guān)于中國Web的大小、形狀和結(jié)構(gòu)[J];計算機研究與發(fā)展;2002年08期

3 韓光濤,張元英;SPECweb99:最新Web服務(wù)器性能基準(zhǔn)測試[J];計算機應(yīng)用研究;2000年04期

4 李昆霖;;淺析性能測試[J];科技信息;2012年09期

5 桑圣洪;胡飛;;性能測試工具LoadRunner的工作機理及關(guān)鍵技術(shù)研究[J];科學(xué)技術(shù)與工程;2007年06期

6 于戈,王國仁,王欣暉,鄭懷遠(yuǎn);一個面向?qū)ο髷?shù)據(jù)庫系統(tǒng)的TPC-C測試與分析[J];軟件學(xué)報;1999年09期

7 白敬培;潘清;馮建峰;王映東;;分布式海量數(shù)據(jù)管理系統(tǒng)Hypertable關(guān)鍵技術(shù)分析[J];網(wǎng)絡(luò)安全技術(shù)與應(yīng)用;2009年05期

相關(guān)碩士學(xué)位論文 前2條

1 劉亮;基于HTTP/S協(xié)議的Web性能測試工具的設(shè)計與實現(xiàn)[D];內(nèi)蒙古大學(xué);2005年

2 劉洋;復(fù)雜業(yè)務(wù)環(huán)境下軟件性能測試模型的設(shè)計與應(yīng)用[D];東北大學(xué)　;2009年

，

本文編號：1880661