視頻云存儲(chǔ)服務(wù)場(chǎng)景下的HDFS負(fù)載均衡工具
發(fā)布時(shí)間:2018-05-02 20:48
本文選題:在線視頻服務(wù) + 云存儲(chǔ); 參考:《小型微型計(jì)算機(jī)系統(tǒng)》2017年02期
【摘要】:在線視頻服務(wù)是互聯(lián)網(wǎng)服務(wù)的重要內(nèi)容,存儲(chǔ)是在線視頻服務(wù)提供的基礎(chǔ).HDFS作為面向通用文件的云存儲(chǔ)系統(tǒng),被很多視頻服務(wù)網(wǎng)站采用,但其負(fù)載均衡工具沒(méi)有考慮利用視頻文件在線播放時(shí)的帶寬消耗特性使集群的帶寬資源得到更充分的利用.為解決這一問(wèn)題,提出視頻存儲(chǔ)場(chǎng)景下的負(fù)載均衡方法 VOBM,它對(duì)視頻文件在線播放時(shí)的帶寬消耗與視頻文件的碼率、數(shù)據(jù)塊大小和訪問(wèn)熱度的關(guān)系進(jìn)行了分析并建立了新的負(fù)載評(píng)估模型,在此基礎(chǔ)上它在負(fù)載方案生成和負(fù)載調(diào)度兩個(gè)環(huán)節(jié)中加入了對(duì)帶寬消耗因素的考慮.在HDFS原有負(fù)載均衡工具的基礎(chǔ)上實(shí)現(xiàn)了該方法,實(shí)驗(yàn)證明該方法能夠有效避免高帶寬消耗數(shù)據(jù)塊的聚集,在高帶寬消耗視頻文件作為服務(wù)訪問(wèn)熱點(diǎn)的實(shí)驗(yàn)場(chǎng)景中,該方法在90%的場(chǎng)景中優(yōu)于原有負(fù)載均衡方法,最高能使數(shù)據(jù)節(jié)點(diǎn)集群中瓶頸節(jié)點(diǎn)的帶寬峰值降低20%.
[Abstract]:Online video service is an important part of Internet service. Storage is the basis of online video service. HDFS, as a cloud storage system oriented to common files, is adopted by many video service websites. However, its load balancing tool does not take into account the bandwidth consumption when video files are played online so that the bandwidth resources of the cluster can be utilized more fully. In order to solve this problem, a load balancing method called VOBM in video storage scenario is proposed, which consumes the bandwidth and the bit rate of the video file when the video file is played online. The relationship between data block size and access heat is analyzed and a new load evaluation model is established. On this basis, the bandwidth consumption factor is considered in the load scheme generation and load scheduling. The method is implemented on the basis of the original load balancing tool of HDFS. Experiments show that the method can effectively avoid the aggregation of high bandwidth consuming data blocks, and in the experimental scenario where high bandwidth consuming video files are used as service access hotspots. This method is superior to the original load balancing method in 90% scenarios, and can reduce the peak bandwidth of bottleneck nodes in the data node cluster by 20%.
【作者單位】: 浙江大學(xué)計(jì)算機(jī)科學(xué)與技術(shù)學(xué)院;浙江省現(xiàn)代服務(wù)業(yè)電子服務(wù)工程技術(shù)研究中心;
【基金】:國(guó)家自然科學(xué)基金項(xiàng)目(61272129)資助 國(guó)家“八六三”高技術(shù)研究發(fā)展計(jì)劃項(xiàng)目(2013AA01A213)資助 教育部?jī)?yōu)秀人才計(jì)劃項(xiàng)目(NCET-12-0491)資助 浙江省杰出青年科學(xué)基金項(xiàng)目(LR13F020002)資助
【分類(lèi)號(hào)】:TP333
【相似文獻(xiàn)】
相關(guān)期刊論文 前1條
1 湯臨春;李心科;潘秋菱;;基于上下文的移動(dòng)Web Services[J];合肥工業(yè)大學(xué)學(xué)報(bào)(自然科學(xué)版);2006年10期
相關(guān)碩士學(xué)位論文 前1條
1 黃曉成;面向視頻云存儲(chǔ)的HDFS負(fù)載均衡工具設(shè)計(jì)與實(shí)現(xiàn)[D];浙江大學(xué);2016年
,本文編號(hào):1835452
本文鏈接:http://sikaile.net/kejilunwen/jisuanjikexuelunwen/1835452.html
最近更新
教材專(zhuān)著