基于MapReduce的城市交通出行分布異常檢測和分析
發(fā)布時間：2019-03-20 14:22
【摘要】：隨著時空軌跡數(shù)據(jù)挖掘的快速發(fā)展，軌跡數(shù)據(jù)異常值檢測已成為數(shù)據(jù)挖掘領(lǐng)域的研究熱點(diǎn)。傳統(tǒng)的異常檢測方法在檢測軌跡數(shù)據(jù)異常值時很多都基于歐式空間環(huán)境，將異常值表示為遠(yuǎn)離大部分一定距離的點(diǎn)。但在交通事件應(yīng)急響應(yīng)等方面的實(shí)際應(yīng)用中，交通出行分布異常的檢測主要通過交通流量的變化進(jìn)行判斷，對傳統(tǒng)異常檢測算法中采用的歐式距離來度量異常的方法不再適用。此外，交通軌跡數(shù)據(jù)量龐大，使用傳統(tǒng)的、單機(jī)運(yùn)行的異常檢測方法運(yùn)行效率也較低。在本文中，利用MapReduce分布式并行計算框架，提出了一種基于MapReduce的分布式并行城市交通出行分布異常檢測和分析算法。具體工作如下： (1)為了能更好的描述交通出行分布情況，本文提出了一種基于小區(qū)交通流量的城市交通出行分布模型。該模型較為簡單且容易理解，能夠從宏觀上展現(xiàn)整個城市的交通出行分布狀況。 (2)針對交通出行分布異常檢測問題，本文結(jié)合交通領(lǐng)域知識，在城市交通流量分布模型的基礎(chǔ)之上提出了基于小區(qū)交通流量的交通出行分布異常定義，并給出了形式化的表示方法。 (3)在上述工作基礎(chǔ)之上，本文提出了一種基于MapReduce的分布式并行交通出行分布異常檢測和分析算法(MapReduce-Based Distributed ParallelTransportation Distribution Outliers Detection And Analysis Algorithm，簡稱MDPTDODA)。該算法首先對出租車軌跡數(shù)據(jù)進(jìn)行預(yù)處理，然后從出租車軌跡數(shù)據(jù)中提取經(jīng)過小區(qū)之間的交通流量并建立基于小區(qū)交通流量的城市交通出行分布模型。最后整合該分布模型中連續(xù)多天的交通流量，構(gòu)建時間序列集，通過DBSCAN聚類算法和動態(tài)時間扭曲距離(Dynamic Time Warping，，簡稱DTW)進(jìn)行交通出行分布異常檢測，并根據(jù)異常之間的關(guān)系分析異常引起的可能原因。 本文以北京市出租車歷史軌跡數(shù)據(jù)作為原始數(shù)據(jù)，在單機(jī)多核環(huán)境和基于Hadoop的集群環(huán)境下分別對試驗(yàn)算法的單機(jī)版本和分布式并行版本進(jìn)行了實(shí)驗(yàn)，證明了本文提出的MDPTDODA算法在分析處理大量軌跡數(shù)據(jù)時的高效性。同時，本文將實(shí)驗(yàn)結(jié)果與歷史實(shí)際情況進(jìn)行了對比，結(jié)果表明該方法在異常的檢測和分析方面是有效的。
[Abstract]:With the rapid development of spatial-temporal trajectory data mining, anomaly detection of trajectory data has become a hot topic in the field of data mining. Many of the traditional anomaly detection methods are based on the European space environment when detecting the outliers of trajectory data. The outliers are represented as points far away from most of the distance. However, in the practical application of traffic emergency response, the detection of traffic trip distribution anomaly is mainly judged by the change of traffic flow, and the Euclidean distance used in the traditional anomaly detection algorithm is no longer applicable. In addition, the large amount of traffic trajectory data, the use of traditional, single-machine anomaly detection method is also low operating efficiency. In this paper, using MapReduce distributed parallel computing framework, a distributed parallel urban traffic trip anomaly detection and analysis algorithm based on MapReduce is proposed. The specific work is as follows: (1) in order to better describe the traffic travel distribution, this paper proposes a model of urban traffic trip distribution based on community traffic flow. The model is simple and easy to understand, and can show the distribution of traffic travel in the whole city macroscopically. (2) aiming at the problem of abnormal detection of traffic travel distribution, this paper puts forward the definition of traffic trip distribution anomaly based on community traffic flow based on the urban traffic flow distribution model, which is based on the knowledge of traffic field and the urban traffic flow distribution model. A formal representation method is given. (3) on the basis of the above work, this paper proposes a distributed parallel traffic trip distribution anomaly detection and analysis algorithm based on MapReduce (MDPTDODA). For short). The algorithm firstly preprocesses the taxi track data, then extracts the traffic flow from the taxi track data and establishes the urban traffic travel distribution model based on the cell traffic flow. Finally, the traffic flow in the distribution model is integrated for many days, and the time series set is constructed. The traffic trip distribution anomaly detection is carried out by DBSCAN clustering algorithm and dynamic time distortion distance (Dynamic Time Warping, (DTW). According to the relationship between the anomalies, the possible causes of the anomalies are analyzed. Taking the historical track data of Beijing taxi as the original data, this paper makes experiments on the single-machine version and the distributed parallel version of the experimental algorithm in the single-machine multi-core environment and the Hadoop-based cluster environment, respectively. It is proved that the proposed MDPTDODA algorithm is efficient in analyzing and processing a large number of trajectory data. At the same time, the experimental results are compared with the actual situation in history, and the results show that the method is effective in anomaly detection and analysis.
【學(xué)位授予單位】：北京工業(yè)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2014
【分類號】：TP393.08;TP311.13
本文編號：2444295
[Abstract]:With the rapid development of spatial-temporal trajectory data mining, anomaly detection of trajectory data has become a hot topic in the field of data mining. Many of the traditional anomaly detection methods are based on the European space environment when detecting the outliers of trajectory data. The outliers are represented as points far away from most of the distance. However, in the practical application of traffic emergency response, the detection of traffic trip distribution anomaly is mainly judged by the change of traffic flow, and the Euclidean distance used in the traditional anomaly detection algorithm is no longer applicable. In addition, the large amount of traffic trajectory data, the use of traditional, single-machine anomaly detection method is also low operating efficiency. In this paper, using MapReduce distributed parallel computing framework, a distributed parallel urban traffic trip anomaly detection and analysis algorithm based on MapReduce is proposed. The specific work is as follows: (1) in order to better describe the traffic travel distribution, this paper proposes a model of urban traffic trip distribution based on community traffic flow. The model is simple and easy to understand, and can show the distribution of traffic travel in the whole city macroscopically. (2) aiming at the problem of abnormal detection of traffic travel distribution, this paper puts forward the definition of traffic trip distribution anomaly based on community traffic flow based on the urban traffic flow distribution model, which is based on the knowledge of traffic field and the urban traffic flow distribution model. A formal representation method is given. (3) on the basis of the above work, this paper proposes a distributed parallel traffic trip distribution anomaly detection and analysis algorithm based on MapReduce (MDPTDODA). For short). The algorithm firstly preprocesses the taxi track data, then extracts the traffic flow from the taxi track data and establishes the urban traffic travel distribution model based on the cell traffic flow. Finally, the traffic flow in the distribution model is integrated for many days, and the time series set is constructed. The traffic trip distribution anomaly detection is carried out by DBSCAN clustering algorithm and dynamic time distortion distance (Dynamic Time Warping, (DTW). According to the relationship between the anomalies, the possible causes of the anomalies are analyzed. Taking the historical track data of Beijing taxi as the original data, this paper makes experiments on the single-machine version and the distributed parallel version of the experimental algorithm in the single-machine multi-core environment and the Hadoop-based cluster environment, respectively. It is proved that the proposed MDPTDODA algorithm is efficient in analyzing and processing a large number of trajectory data. At the same time, the experimental results are compared with the actual situation in history, and the results show that the method is effective in anomaly detection and analysis.
【學(xué)位授予單位】：北京工業(yè)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2014
【分類號】：TP393.08;TP311.13
【參考文獻(xiàn)】
相關(guān)期刊論文 前2條
1 陳森發(fā),周振國,于棟華;一種動態(tài)OD矩陣估計算法的理論及應(yīng)用[J];東南大學(xué)學(xué)報(自然科學(xué)版);2003年01期
2 姜桂艷;常安德;李琦;伊峰;;基于出租車GPS數(shù)據(jù)的路段平均速度估計模型[J];西南交通大學(xué)學(xué)報;2011年04期
本文編號：2444295
本文鏈接：http://sikaile.net/guanlilunwen/ydhl/2444295.html
最近更新
教材專著
