【摘要】：隨著微博、社交網(wǎng)絡(luò)、移動APP、基于位置的服務(wù)等技術(shù)的興起,全球數(shù)據(jù)出現(xiàn)了爆炸式的增長。大數(shù)據(jù)為數(shù)據(jù)的挖掘和分析提供了豐富的來源,從中挖掘出來的結(jié)果將變得更有意義甚至充滿驚喜。然而面對大數(shù)據(jù)傳統(tǒng)的處理方法和處理能力都漸漸變得力不從心。如何高效的進(jìn)行大數(shù)據(jù)的處理已經(jīng)成為各行各業(yè)爭相研究的熱點問題。本文從影視大數(shù)據(jù)智能分析系統(tǒng)建設(shè)過程中遇到的大數(shù)據(jù)處理的問題出發(fā),分析研究了現(xiàn)有的數(shù)據(jù)處理方案,結(jié)合了云計算和大數(shù)據(jù)處理的相關(guān)處理技術(shù),提出了影視大數(shù)據(jù)處理策略及方法。主要研究主要內(nèi)容包括以下幾個方面,首先對云計算的概念和相關(guān)技術(shù)進(jìn)行研究。選定了 Hadoop開源云平臺作為項目研究的基礎(chǔ)平臺,深入研究了 MapReduce分布式編程模型以及HDFS分布式文件存儲系統(tǒng)。其次深入分析了影視數(shù)據(jù)各部分的業(yè)務(wù)需求,為應(yīng)對不同的數(shù)據(jù)處理需求,充分利用關(guān)系型數(shù)據(jù)庫和Hadoop分布式平臺的優(yōu)點,提出了兩種數(shù)據(jù)處理方案。一種基于關(guān)系型數(shù)據(jù)庫的單節(jié)點數(shù)據(jù)處理,一種基于Hadoop的分布式數(shù)據(jù)處理。最后根據(jù)影視大數(shù)據(jù)智能分析系統(tǒng)不同的數(shù)據(jù)特點和不同的數(shù)據(jù)處理需求,選擇合適的數(shù)據(jù)處理方案進(jìn)行數(shù)據(jù)處理流程的設(shè)計和實現(xiàn),并成功的應(yīng)用于影視大數(shù)據(jù)的處理中,為影視大數(shù)據(jù)分析系統(tǒng)提供了可靠的數(shù)據(jù)支持。
[Abstract]:With the rise of technologies such as Weibo, social networks and mobile APP, location-based services, there has been explosive growth in global data. Big data provides a rich source for data mining and analysis, from which the results will become more meaningful and even full of surprises. However, in the face of big data's traditional treatment methods and processing capacity have gradually become unable to do so. How to deal with big data efficiently has become a hot issue in various industries. Starting from the problems encountered by big data in the construction of big data intelligent analysis system for film and television, this paper analyzes and studies the existing data processing schemes, and combines the related processing technologies of cloud computing and big data processing. The treatment strategy and method of big data in film and television are put forward. The main research contents include the following aspects, first of all, the concept of cloud computing and related technologies are studied. The Hadoop open source cloud platform is selected as the basic platform of project research, and the MapReduce distributed programming model and HDFS distributed file storage system are deeply studied. Secondly, the business requirements of each part of film and television data are deeply analyzed. In order to meet the different data processing requirements and make full use of the advantages of relational database and Hadoop distributed platform, two data processing schemes are proposed. A single node data processing based on relational database and a distributed data processing based on Hadoop. Finally, according to the different data characteristics and different data processing requirements of the film and television big data intelligent analysis system, the appropriate data processing scheme is selected to design and implement the data processing flow, and it is successfully applied to the processing of film and television big data. It provides reliable data support for the film and television big data analysis system.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2016
【分類號】：TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 劉圓;王峰;楊明川;;面向大數(shù)據(jù)的分布式存儲技術(shù)研究[J];電信技術(shù);2015年06期

2 程學(xué)旗;靳小龍;王元卓;郭嘉豐;張鐵贏;李國杰;;大數(shù)據(jù)系統(tǒng)和分析技術(shù)綜述[J];軟件學(xué)報;2014年09期

3 唐世慶;李云龍;田鳳明;胡海榮;;基于Hadoop的云計算與存儲平臺研究與實現(xiàn)[J];四川兵工學(xué)報;2014年08期

4 馮登國;張敏;李昊;;大數(shù)據(jù)安全與隱私保護(hù)[J];計算機學(xué)報;2014年01期

5 陳忻;;暢游數(shù)據(jù)海洋[J];中國建設(shè)信息;2013年06期

6 張松林;;云計算的核心技術(shù)與應(yīng)用實例[J];電子世界;2013年05期

7 張耀祥;;云計算和虛擬化技術(shù)[J];計算機安全;2011年05期

8 詹洪文;;云計算核心技術(shù)及其產(chǎn)業(yè)化淺析[J];硅谷;2011年06期

9 李成華;張新訪;金海;向文;;MapReduce:新型的分布式并行計算編程模型[J];計算機工程與科學(xué);2011年03期

10 胡云;;對云計算技術(shù)及應(yīng)用的研究[J];電腦開發(fā)與應(yīng)用;2011年03期

相關(guān)碩士學(xué)位論文 前1條

1 叢中昌;基于云計算平臺的電信經(jīng)營分析系統(tǒng)中海量數(shù)據(jù)處理研究[D];南京郵電大學(xué);2011年

，

本文編號：2485057