本文關(guān)鍵詞： 流處理 Actor模型 云計算 彈性可伸縮 層路由機制　出處：《計算機研究與發(fā)展》2017年05期 　論文類型：期刊論文

【摘要】：流處理是一種重要的大數(shù)據(jù)應(yīng)用模式,在金融、廣告、物聯(lián)網(wǎng)、社交網(wǎng)絡(luò)等眾多領(lǐng)域得到了廣泛應(yīng)用.在流處理場景中,流數(shù)據(jù)的產(chǎn)生速度往往變化劇烈且不容易預(yù)測.這時,如果數(shù)據(jù)流量峰值超過處理系統(tǒng)的承載能力,可能使得系統(tǒng)運行緩慢甚至崩潰,導(dǎo)致處理作業(yè)失效;如果為了應(yīng)對數(shù)據(jù)流量峰值而過度配置資源,則可能在系統(tǒng)輕載時產(chǎn)生不必要的浪費.為了解決流處理中負(fù)載和資源的匹配問題,流處理系統(tǒng)應(yīng)該具有彈性可伸縮的能力,一方面以高效的方式組織運算資源;另一方面能根據(jù)數(shù)據(jù)流量的實時變化自動地調(diào)整資源使用量.然而,現(xiàn)有的流處理框架對于彈性可伸縮的支持尚很初步.介紹了一種基于Actor模型的彈性可伸縮的流處理框架eSault.eSault首先基于Actor模型將批量的處理單元進(jìn)行分層管理,通過2層路由機制實現(xiàn)了對伸縮性的支持;在此基礎(chǔ)上,設(shè)計一個基于數(shù)據(jù)處理延遲的過載判斷算法和基于數(shù)據(jù)處理速度的輕載判斷算法來指導(dǎo)系統(tǒng)對資源的有效使用,進(jìn)而實現(xiàn)彈性可伸縮的流處理.實驗結(jié)果表明:eSault具有較好的性能,而且能夠很好地實現(xiàn)彈性可伸縮.
[Abstract]:Stream processing is an important big data application model, which has been widely used in many fields, such as finance, advertising, Internet of things, social network and so on. In this case, if the peak value of data flow exceeds the carrying capacity of the processing system, it may make the system run slowly or even collapse, resulting in the failure of processing operations. If resources are over-configured to cope with peak data traffic, unnecessary waste may occur when the system is under light load. In order to solve the problem of load and resource matching in flow processing. The flow processing system should be flexible and scalable. On the one hand, it can organize the computing resources in an efficient way. On the other hand, the resource usage can be automatically adjusted according to the real-time change of data traffic. The support of current flow processing framework for elastic scalability is very preliminary. This paper introduces an elastic scalable flow processing framework based on Actor model eSault.eSault based on Acto at first. R model delaminate the batch processing units. Through the two-layer routing mechanism, the scalability support is realized. On this basis, we design an overload judgment algorithm based on data processing delay and a light-load judgment algorithm based on data processing speed to guide the effective use of resources. The experimental results show that: e Sault has good performance and is able to achieve elastic scalability.
【作者單位】： 高可信軟件技術(shù)教育部重點實驗室(北京大學(xué));北京大學(xué)(天津濱海)新一代信息技術(shù)研究院;
【基金】：國家“八六三”高技術(shù)研究發(fā)展計劃基金項目(2015AA01A202) 國家“九七三”重點基礎(chǔ)研究計劃基金項目(2011CB302604) 國家自然科學(xué)基金項目(61272154,61421091) 百度云服務(wù)開放平臺示范項目(2015年)~~
【分類號】：TP311.13
【正文快照】： This work was supported by the National High Technology Research and Development Program of China(863 Program)(2015AA01A202),the National Basic Research Program of China(973Program)(2011CB302604),the National Natural ScienceFoundation of China(G61272154,

【相似文獻(xiàn)】

相關(guān)期刊論文 前8條

1 董哲,劉琳,田籟聲;基于ACTOR模型的并發(fā)面向?qū)ο笳Z言AC++[J];軟件學(xué)報;1997年03期

2 唐昊;陳棟;周雷;吳玉華;;SMDP基于Actor網(wǎng)絡(luò)的統(tǒng)一NDP方法[J];控制與決策;2007年02期

3 吳眾欣;錢德沛;黃泳翔;;基于軟件管道Actor模型的BPEL流程轉(zhuǎn)化研究[J];計算機技術(shù)與發(fā)展;2009年07期

4 王尋羽,朱淼良,徐文淵;基于Actor的虛擬環(huán)境系統(tǒng)研究[J];計算機輔助設(shè)計與圖形學(xué)學(xué)報;2001年01期

5 林昊;;在Java中使用協(xié)程[J];程序員;2010年03期

6 馮嵩;;.net平臺下的Agent遷移設(shè)計與實現(xiàn)[J];電腦與電信;2008年07期

7 ;Event-based Implicit Invocation Decentralized in Ada[J];Wuhan University Journal of Natural Sciences;2001年Z1期

8 ;[J];;年期

，

本文編號：1455317