本文選題：智能交通　切入點(diǎn)：Hadoop　出處：《揚(yáng)州大學(xué)》2015年碩士論文

【摘要】：隨著智能交通基礎(chǔ)建設(shè)的快速發(fā)展,城鎮(zhèn)居民收入水平逐步提高,城市汽車擁有量大幅度增加。遍布每個(gè)城市道路的感應(yīng)線圈、卡口斷面系統(tǒng),能夠及時(shí)地采集、記錄、匯總并上傳監(jiān)控?cái)?shù)據(jù)。但是由于城市道路交通流存在著數(shù)據(jù)量大、實(shí)時(shí)性高等特點(diǎn),傳統(tǒng)的數(shù)據(jù)存儲(chǔ)與處理技術(shù)存在著數(shù)據(jù)結(jié)構(gòu)與數(shù)據(jù)存儲(chǔ)容量無法靈活擴(kuò)展、分布式并行數(shù)據(jù)挖掘難、高容錯(cuò)恢復(fù)能力差等問題。如何將海量的交通流數(shù)據(jù)實(shí)時(shí)地上傳、匯總和存儲(chǔ)利用,以及如何對(duì)數(shù)據(jù)進(jìn)行統(tǒng)計(jì)挖掘成為一個(gè)較大的難題。以Hadoop為代表的大數(shù)據(jù)技術(shù)成為解決這一系列問題的有效手段之一�；诂F(xiàn)階段城市交通發(fā)展帶來的數(shù)據(jù)存儲(chǔ)與分析等突出問題,本文通過對(duì)基于Hadoop的MapReduce、HBase等大數(shù)據(jù)技術(shù)的研究,提出了相應(yīng)的解決方案,其主要研究工作和成果如下：(1)本文提出了基于Hadoop的交通流數(shù)據(jù)存儲(chǔ)與分析總體架構(gòu)。將架構(gòu)分為5個(gè)層面：數(shù)據(jù)采集層、硬件平臺(tái)層、數(shù)據(jù)存儲(chǔ)與計(jì)算層、挖掘分析層和應(yīng)用服務(wù)層,同時(shí)研究與設(shè)計(jì)了節(jié)點(diǎn)在故障或宕機(jī)情況下,Hadoop集群具有高容錯(cuò)恢復(fù)能力的可用性方案。(2)本文提出了基于HBase的海量交通流數(shù)據(jù)分布式存儲(chǔ)方案。根據(jù)交通流數(shù)據(jù)特點(diǎn)與處理應(yīng)用需求,設(shè)計(jì)了可解決“熱點(diǎn)”問題的交通流數(shù)據(jù)表行健結(jié)構(gòu)。同時(shí)研究了HBase的協(xié)處理器,設(shè)計(jì)了用于針對(duì)列查詢的快速數(shù)據(jù)檢索的二級(jí)索引表。(3)本文還根據(jù)交通車流量與密度的關(guān)系,設(shè)計(jì)了流量與密度計(jì)算模型,提出了基于MapReduce的流量密度計(jì)算的并行化實(shí)現(xiàn),解決了海量交通流數(shù)據(jù)情況下的流量、密度快速計(jì)算難題。同時(shí),采用K近鄰非參數(shù)回歸算法來預(yù)測(cè)短時(shí)交通流,通過對(duì)K近鄰狀態(tài)向量、距離度量方式、近鄰個(gè)數(shù)以及預(yù)測(cè)算法的選擇及研究,提出了基于MapReduce的KNN預(yù)測(cè)短時(shí)交通流的并行化實(shí)現(xiàn),加快K最近鄰算法的搜索速度,實(shí)現(xiàn)對(duì)短時(shí)交通流的定時(shí)預(yù)測(cè)。(4)最后,根據(jù)總體架構(gòu)應(yīng)用層需求,基于Hadoop平臺(tái),構(gòu)建并實(shí)現(xiàn)了城市道路交通流數(shù)據(jù)分析系統(tǒng)。本文對(duì)系統(tǒng)進(jìn)行了詳細(xì)的功能模塊設(shè)計(jì),并實(shí)現(xiàn)了對(duì)交通流量進(jìn)行實(shí)時(shí)監(jiān)測(cè)、海量數(shù)據(jù)分析的圖形化展示等功能。
[Abstract]:With the rapid development of intelligent transportation infrastructure, the income level of urban residents has gradually increased, and the number of urban car ownership has increased significantly.Induction coil and bayonet section system all over every city road can collect, record, collect and upload monitoring data in time.However, due to the characteristics of large amount of data and high real-time performance in urban road traffic flow, the traditional data storage and processing technology can not extend data structure and storage capacity flexibly, and distributed parallel data mining is difficult.Poor recovery ability of high fault tolerance and so on.How to upload, aggregate, store and utilize massive traffic flow data in real time, and how to mine the data statistically has become a big problem.Big data technology, represented by Hadoop, has become one of the effective means to solve this series of problems.Based on the outstanding problems of data storage and analysis brought about by the development of urban traffic at present, this paper puts forward the corresponding solutions through the research of big data technology such as MapReduceHBase based on Hadoop.The main research work and results are as follows: (1) this paper proposes an overall framework of traffic flow data storage and analysis based on Hadoop.The architecture is divided into five layers: data acquisition layer, hardware platform layer, data storage and computing layer, mining analysis layer and application service layer.At the same time, we study and design the availability scheme of Hadoop cluster with high fault-tolerant recovery ability in the event of failure or downtime) this paper proposes a distributed storage scheme for massive traffic flow data based on HBase.According to the characteristics of traffic flow data and the requirement of application, the traffic flow data table is designed to solve the "hot spot" problem.At the same time, the coprocessor of HBase is studied, and a two-level index table for fast data retrieval for column query is designed. In this paper, according to the relationship between traffic flow and density, the calculation model of traffic flow and density is also designed.A parallel implementation of traffic density calculation based on MapReduce is proposed, which solves the problem of fast calculation of traffic density in the case of massive traffic flow data.At the same time, the K-nearest neighbor nonparametric regression algorithm is used to predict the short-term traffic flow. The selection and research of K-nearest neighbor state vector, distance measurement, number of nearest neighbors and prediction algorithm are carried out.In this paper, the parallel implementation of short time traffic flow prediction with KNN based on MapReduce is proposed, which speeds up the search speed of K nearest neighbor algorithm, and realizes the timing prediction of short time traffic flow. Finally, according to the requirements of the application layer of the overall architecture, it is based on Hadoop platform.The data analysis system of urban road traffic flow is constructed and implemented.In this paper, the function module of the system is designed in detail, and the functions of real-time monitoring of traffic flow and graphical display of mass data analysis are realized.
【學(xué)位授予單位】：揚(yáng)州大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2015
【分類號(hào)】：U495;TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 梁軻;譚建軍;李英遠(yuǎn);;一種基于MapReduce的短時(shí)交通流預(yù)測(cè)方法[J];計(jì)算機(jī)工程;2015年01期

2 陸婷;房俊;喬彥克;;基于HBase的交通流數(shù)據(jù)實(shí)時(shí)存儲(chǔ)系統(tǒng)[J];計(jì)算機(jī)應(yīng)用;2015年01期

3 謝海紅;戴許昊;齊遠(yuǎn);;短時(shí)交通流預(yù)測(cè)的改進(jìn)K近鄰算法[J];交通運(yùn)輸工程學(xué)報(bào);2014年03期

4 焦冬冬;徐新國(guó);;一種基于HBase的海量微博數(shù)據(jù)高效存儲(chǔ)方案[J];微型機(jī)與應(yīng)用;2014年11期

5 李建國(guó);;智能交通發(fā)展中的大數(shù)據(jù)分析[J];硅谷;2014年06期

6 司文;;hadoop技術(shù)在交通卡口數(shù)據(jù)管理中的應(yīng)用[J];電子技術(shù)與軟件工程;2013年17期

7 閆永剛;馬廷淮;王建;;KNN分類算法的MapReduce并行化實(shí)現(xiàn)[J];南京航空航天大學(xué)學(xué)報(bào);2013年04期

8 陳美;;大數(shù)據(jù)在公共交通中的應(yīng)用[J];圖書與情報(bào);2012年06期

9 朱晨杰;楊永麗;;基于MapReduce的BP神經(jīng)網(wǎng)絡(luò)算法研究[J];微型電腦應(yīng)用;2012年10期

10 于濱;鄔珊華;王明華;趙志宏;;K近鄰短時(shí)交通流預(yù)測(cè)模型[J];交通運(yùn)輸工程學(xué)報(bào);2012年02期

，

本文編號(hào)：1720181