本文選題：融合通信　切入點：分布式　出處：《中國科學院大學(中國科學院沈陽計算技術研究所)》2017年碩士論文　論文類型：學位論文

【摘要】：互聯(lián)網(wǎng)OTT業(yè)務和移動互聯(lián)網(wǎng)應用得到了快速的普及,使人們對信息獲取的方式開始變得多元化,對通信服務的質(zhì)量要求也越來越高,傳統(tǒng)意義上基于語音通話或者短信技術層面上的通信業(yè)務已經(jīng)無法滿足人們的日常需求,通信業(yè)務也逐漸向包含語音、視頻等的多媒體通信方向發(fā)展,因此而形成的將傳統(tǒng)通信技術與互聯(lián)網(wǎng)信息技術相融合的融合通信技術,成為當下計算機應用領域的一個研究熱點。在融合通信系統(tǒng)數(shù)據(jù)存取方面,需要對即時消息、文檔、組織架構(gòu)等關鍵數(shù)據(jù)提供完善的存儲引擎機制,而傳統(tǒng)的基于關系數(shù)據(jù)庫或者基于傳統(tǒng)文件系統(tǒng)的存儲方式,在存儲數(shù)據(jù)安全性、獲取數(shù)據(jù)效率以及后續(xù)數(shù)據(jù)挖掘與分析等方面都存在不滿足的情況,因此亟需一種能夠更好的滿足需求的存儲服務模式。隨著Hadoop技術和Hadoop相關子系統(tǒng)的發(fā)展成熟,分布式存儲的優(yōu)勢日益明顯,本文在分析HDFS、MapReduce并行計算框架以及HBase/Hive體系結(jié)構(gòu)和各自特點的基礎之上,提出一種基于HBase-Hive集成設計的存儲引擎設計方案,以此來滿足融合通信系統(tǒng)對數(shù)據(jù)安全性、數(shù)據(jù)獲取實時性和可靠性等方面的要求,同時充分研究數(shù)據(jù)挖掘的基礎理論以及K-means、PAM聚類算法,結(jié)合MapReduce并行計算模型設計并實現(xiàn)了改進型K-means聚類算法,以此作為融合通信對數(shù)據(jù)挖掘需求的解決方案。在論文結(jié)構(gòu)上,本文首先詳細分析了課題研究的背景、現(xiàn)狀、意義和相關基礎技術,結(jié)合PAM算法改進K-means算法,然后設計并實現(xiàn)了分布式存儲引擎的各個功能模塊,最后通過對比試驗進行功能和性能測試以及針對算法的仿真實驗,驗證了分布式存儲引擎在融合通信系統(tǒng)中的可行性和合理性。
[Abstract]:Internet OTT services and mobile Internet applications have been rapidly popularized, making people become more and more diverse in the way of obtaining information, and the quality of communication services is becoming more and more demanding. The traditional communication service based on voice call or short message technology has been unable to meet the daily needs of people, and the communication service has gradually developed towards multimedia communication including voice, video and so on. Therefore, the fusion communication technology that combines traditional communication technology with Internet information technology has become a research hotspot in the field of computer application. In the aspect of data access of fusion communication system, instant messaging and documents are needed. Organization structure and other key data provide perfect storage engine mechanism, while traditional storage methods based on relational database or traditional file system are used to store data security. Data acquisition efficiency and subsequent data mining and analysis are not satisfied, so it is urgent to have a storage service mode that can better meet the requirements. With the development of Hadoop technology and Hadoop related subsystems, The advantages of distributed storage are becoming more and more obvious. Based on the analysis of HDFS MapReduce parallel computing framework, HBase/Hive architecture and their respective characteristics, a storage engine design scheme based on HBase-Hive integrated design is proposed. In order to meet the requirements of data security, real-time and reliability of data acquisition, the basic theory of data mining and K-means-PAM clustering algorithm are studied. Combined with MapReduce parallel computing model, the improved K-means clustering algorithm is designed and implemented as a solution to the data mining requirements of fusion communication. In the structure of the thesis, the background and present situation of the research are analyzed in detail. Significance and related basic technology, combined with PAM algorithm to improve K-means algorithm, and then designed and implemented each functional module of the distributed storage engine. Finally, the function and performance of the distributed storage engine were tested by contrast experiment and the simulation experiment for the algorithm was carried out. The feasibility and rationality of distributed storage engine in fusion communication system are verified.
【學位授予單位】：中國科學院大學(中國科學院沈陽計算技術研究所)
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP311.13

【參考文獻】

相關期刊論文 前4條

1 卜旭松;劉立波;石磊;;基于PAM和簇閾值的改進K-Means聚類算法[J];湖北工程學院學報;2015年03期

2 趙彥榮;王偉平;孟丹;張書彬;李均;;基于Hadoop的高效連接查詢處理算法CHMJ[J];軟件學報;2012年08期

3 梁建武;田野;;一種分布式的K-means聚類算法[J];現(xiàn)代電子技術;2010年10期

4 柴小卉;數(shù)據(jù)挖掘在金融領域中的應用及發(fā)展趨勢[J];華南金融電腦;2004年09期

相關碩士學位論文 前7條

1 高大月;基于Hadoop的數(shù)據(jù)倉庫引擎的設計與實現(xiàn)[D];北京交通大學;2015年

2 何健偉;基于Hadoop的數(shù)據(jù)挖掘算法研究與實現(xiàn)[D];北京郵電大學;2015年

3 劉珂;基于Hadoop平臺的大數(shù)據(jù)遷移與查詢方法研究及應用[D];武漢理工大學;2014年

4 馮曉普;HBase存儲的研究與應用[D];北京郵電大學;2014年

5 呂明育;Hadoop架構(gòu)下數(shù)據(jù)挖掘與數(shù)據(jù)遷移系統(tǒng)的設計與實現(xiàn)[D];上海交通大學;2013年

6 朱琨;基于P2P的分布式存儲系統(tǒng)的研究與實現(xiàn)[D];電子科技大學;2010年

7 黃關山;基于聚類規(guī)則挖掘的教學分析系統(tǒng)設計與實現(xiàn)[D];東南大學;2005年

，

本文編號：1560532