本文選題：聯(lián)機(jī)分析處理 + HOLAP　； 參考：《電子科技大學(xué)》2016年碩士論文

【摘要】：近年來(lái),多維數(shù)據(jù)查詢(xún)聯(lián)機(jī)分析處理技術(shù)(Online Analytical Processing,OLAP)越來(lái)越重要�；贠LAP的多維分析技術(shù)成為企業(yè)管理人員重要的決策依據(jù)。目前,針對(duì)OLAP的研究都是面向單一數(shù)據(jù)模型的存儲(chǔ)處理和相應(yīng)OLAP查詢(xún)性能上的優(yōu)化。單一數(shù)據(jù)組織模式的基于關(guān)系數(shù)據(jù)庫(kù)的ROLAP(Relational-OLAP)和基于多維數(shù)據(jù)庫(kù)的MOLAP(Multidimensional-OLAP),無(wú)法滿足在不同規(guī)模級(jí)別數(shù)據(jù)集下異構(gòu)數(shù)據(jù)模型和低延遲的多維查詢(xún)需求。針對(duì)以上問(wèn)題,本文從不同數(shù)據(jù)組織模型的查詢(xún)規(guī)劃、查詢(xún)解釋、緩存查詢(xún)優(yōu)化機(jī)制等方面改進(jìn),設(shè)計(jì)和實(shí)現(xiàn)了一個(gè)可擴(kuò)展性和高效性的分布式混合型聯(lián)機(jī)分析處理(Hybrid-OLAP,HOLAP)系統(tǒng)。該系統(tǒng)旨在解決不同規(guī)模級(jí)別數(shù)據(jù)集的多維查詢(xún),根據(jù)不同多維組織的實(shí)現(xiàn)模式作出高效合理的查詢(xún)處理�；谠撓到y(tǒng)下的研究主要包括以下四個(gè)方面的內(nèi)容:第一,針對(duì)傳統(tǒng)ROLAP系統(tǒng)無(wú)法高效地解決大規(guī)模數(shù)據(jù)集的多維分析問(wèn)題,提出了一個(gè)能夠在Hadoop環(huán)境下,滿足不同規(guī)模級(jí)別數(shù)據(jù)集進(jìn)行快速多維查詢(xún)分析,同時(shí)支持Hive的MDX(Multidimensional Expressions)查詢(xún)解釋和聚集方法,以及基于Hbase預(yù)計(jì)算緩存機(jī)制的多維查詢(xún)優(yōu)化方法的HOLAP系統(tǒng)架構(gòu)。第二,針對(duì)大規(guī)模數(shù)據(jù)集上的Hive多維查詢(xún)優(yōu)化,通過(guò)一種構(gòu)建Hbase立方體緩存的分段逐層降維聚集算法(S-Redu-D-A),研究了從類(lèi)似關(guān)系型數(shù)據(jù)庫(kù)Hive到Nosql數(shù)據(jù)庫(kù)中,Hbase數(shù)據(jù)模型的形式化方法(Hsql-To-Nosql Formalized Model,Hs-Nos-FM)。提出并驗(yàn)證了滿足HOLAP高效地形式化多維立方體(Format Multi Cube,F-M-Cube)數(shù)據(jù)存儲(chǔ)模型,在大規(guī)模數(shù)據(jù)集多維查詢(xún)上表現(xiàn)出良好的性能。第三,針對(duì)兩種查詢(xún)計(jì)劃,通過(guò)實(shí)時(shí)性要求、數(shù)據(jù)規(guī)模、維度基數(shù)、存儲(chǔ)空間、多表連接、查詢(xún)頻率等指標(biāo)進(jìn)行查詢(xún)規(guī)劃計(jì)算分析;提出了包含權(quán)限控制、查詢(xún)監(jiān)聽(tīng)、查詢(xún)分析和查詢(xún)分配的查詢(xún)規(guī)劃工作流程。通過(guò)對(duì)不同規(guī)模數(shù)據(jù)、不同多維查詢(xún)的執(zhí)行時(shí)間對(duì)比分析,有效地驗(yàn)證了基于HOLAP系統(tǒng)架構(gòu)的查詢(xún)規(guī)劃方法,在常見(jiàn)OLAP多維查詢(xún)中表現(xiàn)出良好的性能。最后,本文通過(guò)HOLAP系統(tǒng)架構(gòu)下的查詢(xún)規(guī)劃方法、查詢(xún)解釋機(jī)制、形式化多維立方體構(gòu)建方法、聚集緩存機(jī)制、支持Hive的MDX查詢(xún),并嵌入形式化方法的構(gòu)建算法進(jìn)行詳細(xì)設(shè)計(jì)和實(shí)現(xiàn)。經(jīng)過(guò)測(cè)試,本系統(tǒng)具有良好的性能,達(dá)到了預(yù)期的設(shè)計(jì)目標(biāo)。
[Abstract]:In recent years, online Analytical processing technology (OLAP) is becoming more and more important. Multidimensional analysis technology based on OLAP has become an important decision basis for enterprise managers. At present, the research of OLAP is focused on the storage and processing of single data model and the optimization of OLAP query performance. The single data organization model based on relational database relation al-OLAP) and the multidimensional database based model Multidimensional-OLAPP can not meet the requirements of heterogeneous data model and low latency multidimensional query under different scale data sets. Aiming at the above problems, this paper improves the query planning, query interpretation and cache query optimization mechanism of different data organization models, and designs and implements a distributed hybrid on-line analytical processing system named hybrid-OLAPHLAPP. The purpose of the system is to solve the multi-dimensional query of data sets of different scales and to make efficient and reasonable query processing according to the implementation mode of different multidimensional organizations. The research based on this system mainly includes the following four aspects: first, aiming at the traditional ROLAP system can not solve the multidimensional analysis problem of large-scale data sets efficiently, a new method is proposed, which can be used in the Hadoop environment. At the same time, it supports the MDX(Multidimensional expressions of Hive query interpretation and aggregation method, and the HOLAP system architecture based on the Hbase prediction cache mechanism of multidimensional query optimization method. Second, for Hive multidimensional query optimization on large data sets, In this paper, we study the formal method of Hbase data model from similar relational database (Hive) to Nosql database (Nosql) through a piecewise hierarchical dimensionality reduction aggregation algorithm (S-Redu-D-An), which is used to construct Hbase cube cache. The formal method is Hsql-To-Nosql Formalized Model-Hs-Nos-FMN. This paper presents and verifies the efficient formative data storage model of multi-dimensional cube format Multi F-M-Cubesatisfying HOLAP, and shows good performance on multidimensional query of large data sets. Third, for two query plans, through real-time requirements, data size, dimensional cardinality, storage space, multi-table join, query frequency and other indicators for query planning and calculation analysis; proposed including authority control, query monitoring, Query analysis and query allocation of query planning workflow. By comparing and analyzing the execution time of different scale data and multidimensional query, the query planning method based on HOLAP system architecture is validated effectively, and it shows good performance in common OLAP multidimensional query. Finally, through the query planning method, query interpretation mechanism, formalization of multidimensional cube construction method, gathering cache mechanism, this paper supports MDX query of Hive. And embed formal method to build the algorithm for detailed design and implementation. After testing, the system has good performance and achieves the expected design goal.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2016
【分類(lèi)號(hào)】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 熊寧;;大數(shù)據(jù)分析的分布式MOLAP技術(shù)[J];信息技術(shù)與信息化;2015年02期

2 王會(huì)舉;覃雄派;王珊;張延松;李芙蓉;;面向大規(guī)模機(jī)群的可擴(kuò)展OLAP查詢(xún)技術(shù)[J];計(jì)算機(jī)學(xué)報(bào);2015年01期

3 郭朝鵬;王智;韓峰;張一川;宋杰;;HaoLap:基于Hadoop的海量數(shù)據(jù)OLAP系統(tǒng)[J];計(jì)算機(jī)研究與發(fā)展;2013年S1期

4 張延松;焦敏;王占偉;王珊;周p，

本文編號(hào)：1907639