【摘要】：目前云計算環(huán)境下海量數(shù)據(jù)的查詢分析以批處理、線下計算為主,很難達(dá)到聯(lián)機(jī)性、即席性和交互性。數(shù)據(jù)立方體是數(shù)據(jù)倉庫、商業(yè)智能中重要的數(shù)據(jù)模型,其數(shù)據(jù)單元之間根據(jù)上卷、下鉆關(guān)系構(gòu)成了格結(jié)構(gòu)數(shù)據(jù)。為了提高查詢分析性能,本文基于數(shù)據(jù)立方體格模型及分布式內(nèi)存架構(gòu)研究在線、高效的聯(lián)機(jī)分析處理技術(shù)。 本文主要在以下兩個方面進(jìn)行研究： (1)視格結(jié)構(gòu)數(shù)據(jù)為圖結(jié)構(gòu)數(shù)據(jù),以格結(jié)構(gòu)數(shù)據(jù)的統(tǒng)計特性和規(guī)律為切入點(diǎn),利用實(shí)驗(yàn)統(tǒng)計方法、復(fù)雜網(wǎng)絡(luò)等經(jīng)典解析模型、格的概念層次結(jié)構(gòu),研究格結(jié)構(gòu)數(shù)據(jù)的模型；在此基礎(chǔ)上,結(jié)合當(dāng)前的圖劃分技術(shù)研究格結(jié)構(gòu)數(shù)據(jù)在多個節(jié)點(diǎn)的劃分及存儲,從而消耗較小的通訊代價并使得集群負(fù)載均衡。 (2)分層封閉立方體是封閉立方體模型的擴(kuò)展,是對數(shù)據(jù)立方體的一種語義壓縮,能夠有效地減少數(shù)據(jù)立方體的存儲空間。對封閉數(shù)據(jù)立方體使用數(shù)組和格兩種結(jié)構(gòu)存儲,利用封閉單元的層次信息以及封閉單元之間元組覆蓋關(guān)系,研究兩種結(jié)構(gòu)下分層封閉立方體的分布式存儲、查詢方法。 最終構(gòu)建基于內(nèi)存的分布式計算框架Spark實(shí)現(xiàn)了OLAP查詢原型系統(tǒng),并進(jìn)行了實(shí)驗(yàn)驗(yàn)證和分析。
[Abstract]:At present, the query analysis of massive data in cloud computing environment is mainly based on batch processing, offline computing, it is difficult to achieve online, ad hoc and interactive. Data cube is an important data model in business intelligence. In order to improve the performance of query analysis, this paper studies online and efficient OLAP technology based on data cube model and distributed memory architecture. This paper mainly studies in the following two aspects: (1) taking the lattice structure data as the graph structure data, taking the statistical characteristics and rules of the lattice structure data as the breakthrough point, using the experimental statistical method, the complex network and other classical analytical models. The conceptual hierarchy of lattice, the model of lattice structure data, and the partition and storage of lattice structure data in multiple nodes are studied based on the current graph partitioning technique. Therefore, it consumes less communication cost and makes cluster load balance. (2) hierarchical closed cube is an extension of closed cube model, and it is a semantic compression of data cube. It can effectively reduce the storage space of data cube. Two kinds of structures are used to store closed data cubes: array and lattice. Using the hierarchical information of closed units and the tuple overlay relationship between closed cells, the distributed storage and query method of hierarchical closed cubes under two structures are studied. Finally, a distributed computing framework based on memory, Spark, is built to implement the OLAP query prototype system, and the experimental verification and analysis are carried out.
【學(xué)位授予單位】：昆明理工大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2014
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前4條

1 符青云;劉心松;;基于協(xié)同緩存的分布式數(shù)據(jù)庫更新機(jī)制研究[J];計算機(jī)工程;2007年07期

2 劉建;孫曉輝;倪宏;;一種基于DHT的Web緩存共享方法[J];計算機(jī)應(yīng)用研究;2008年12期

3 李盛恩,王珊;封閉數(shù)據(jù)立方體技術(shù)研究[J];軟件學(xué)報;2004年08期

4 秦秀磊;張文博;魏峻;王偉;鐘華;黃濤;;云計算環(huán)境下分布式緩存技術(shù)的現(xiàn)狀與挑戰(zhàn)[J];軟件學(xué)報;2013年01期

，

本文編號：2204107