本文選題：智慧交通 + 檢索　； 參考：《浙江大學(xué)》2017年碩士論文

【摘要】：隨著城市化進(jìn)程加快,城市人口迅速增加,帶來了日益嚴(yán)重的交通安全問題�！爸悄芙煌ā笨梢酝ㄟ^城市監(jiān)控所采集的大量數(shù)據(jù),對交通情況、城市安全進(jìn)行分析,協(xié)助城市管理者管理城市的交通與安全。HBase數(shù)據(jù)庫可以大規(guī)模存儲車輛信息,但是無法快速實(shí)時進(jìn)行檢索。本研究主要面向?qū)Base中存儲的過車記錄進(jìn)行快速實(shí)時檢索,提出通過降低網(wǎng)絡(luò)開銷、硬盤讀取開銷的優(yōu)化方法。首先,在HBase中建立二級索引降低檢索中掃描的數(shù)據(jù)量,計(jì)算二級索引來獲得符合檢索條件的記錄。實(shí)驗(yàn)結(jié)果表明,返回第一條過車記錄的檢索響應(yīng)時間相比無二級索引時降低45倍。其次,二級索引從存儲端傳輸至客戶端的網(wǎng)絡(luò)傳輸開銷時間占總檢索時間的20%。為了降低網(wǎng)絡(luò)傳輸開銷,基于近數(shù)據(jù)計(jì)算理論,通過HBase協(xié)處理器構(gòu)建加速框架將二級索引的獲取和計(jì)算移動到存儲節(jié)點(diǎn)進(jìn)行。在集成了該加速框架后,網(wǎng)絡(luò)開銷降低了 12倍,檢索響應(yīng)時間加速比為1.4。最后,在檢索時發(fā)現(xiàn)存儲節(jié)點(diǎn)從硬盤讀取二級索引的時間占總時間開銷的70%。為了降低硬盤讀取時間開銷,根據(jù)二級索引離散度的不同,組合多種無損壓縮算法對二級索引進(jìn)行壓縮來降低硬盤讀取時間,并在HBase協(xié)處理器中實(shí)現(xiàn)壓縮后索引的計(jì)算。實(shí)驗(yàn)結(jié)果表明,硬盤讀取時間降低73%,檢索響應(yīng)時間降低 80%。
[Abstract]:With the acceleration of urbanization and the rapid increase of urban population, traffic safety problems are becoming more and more serious. "Intelligent Transportation" can analyze the traffic situation and urban safety through a large amount of data collected by urban monitoring, and help city managers manage the city's traffic and safety. HBase database can store vehicle information on a large scale. But there is no quick and real-time retrieval. This research mainly aims at the fast real-time retrieval of the passing records stored in HBase, and proposes an optimization method to reduce the network overhead and the hard disk reading overhead. Firstly, a secondary index is established in HBase to reduce the amount of data scanned in the retrieval process, and the secondary index is calculated to obtain the records that meet the retrieval criteria. The experimental results show that the retrieval response time of returning the first passing record is 45 times lower than that without the second level index. Secondly, the network transmission overhead from storage to client accounts for 20% of the total retrieval time. In order to reduce the network transmission overhead, based on the theory of near data computing, an accelerated framework is constructed by HBase coprocessor to move the acquisition and computation of the secondary index to the storage node. With the integration of the acceleration framework, the network overhead is reduced by 12 times, and the retrieval response time speedup is 1.4. Finally, it is found that the storage node reads the secondary index from the hard disk in 70% of the total time cost. In order to reduce the read time cost of the hard disk, according to the difference of the dispersion of the secondary index, several lossless compression algorithms are combined to compress the secondary index to reduce the read time of the hard disk, and the computation of the compressed index is realized in the HBase coprocessor. The experimental results show that the hard disk reading time is reduced by 73 and the retrieval response time is reduced by 80 percent.
【學(xué)位授予單位】：浙江大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：U495;U492.8

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 奚雪峰;周國棟;;面向自然語言處理的深度學(xué)習(xí)研究[J];自動化學(xué)報;2016年10期

2 雷力;錢斌海;郭俊;顧雄禮;劉鵬;;集成I/O硬件壓縮加速器的Hadoop系統(tǒng)結(jié)構(gòu)[J];計(jì)算機(jī)工程與科學(xué);2016年08期

3 楊正;;基于大數(shù)據(jù)架構(gòu)的智能交通解決方案[J];北京聯(lián)合大學(xué)學(xué)報;2014年04期

4 巫細(xì)波;楊再高;;智慧城市理念與未來城市發(fā)展[J];城市發(fā)展研究;2010年11期

5 賈利民,李平,秦勇,史天運(yùn),聶阿新;中國鐵路智能運(yùn)輸系統(tǒng)的體系框架[J];中國鐵路;2003年09期

6 聶阿新;鐵路智能運(yùn)輸系統(tǒng)應(yīng)用前景、框架體系和關(guān)鍵技術(shù)研究[J];中國鐵道科學(xué);2002年02期

，

本文編號：1795863