【摘要】：隨著信息化技術(shù)的發(fā)展和應(yīng)用,各行業(yè)產(chǎn)生的數(shù)據(jù)呈爆炸式增長。傳統(tǒng)數(shù)據(jù)庫技術(shù)不能很好地解決海量數(shù)據(jù)、高并發(fā)、快速響應(yīng)、可擴(kuò)展性等大數(shù)據(jù)應(yīng)用問題。因此如何高效地存儲(chǔ)和管理這些數(shù)據(jù),是目前亟需解決的問題。云計(jì)算的經(jīng)濟(jì)性、可擴(kuò)展性、容錯(cuò)性等特點(diǎn)使其成為大數(shù)據(jù)管理的支撐技術(shù)。隨著云計(jì)算的廣泛應(yīng)用,數(shù)據(jù)中心的數(shù)量和規(guī)�？焖僭鲩L,電費(fèi)開銷已經(jīng)超過了硬件設(shè)備本身的購置費(fèi)用,且仍處于持續(xù)增長狀態(tài)�？焖僭鲩L的能量消耗同時(shí)加劇全球能源危機(jī)和環(huán)境污染。因此研究云環(huán)境下能量高效的數(shù)據(jù)管理技術(shù)刻不容緩。云環(huán)境下能量高效的任務(wù)調(diào)度技術(shù)是云環(huán)境下能量高效的數(shù)據(jù)管理技術(shù)的重要組成部分,目的是通過任務(wù)調(diào)度的方法降低用于任務(wù)處理的節(jié)點(diǎn)在任務(wù)處理過程中的能量消耗。本文分別從單節(jié)點(diǎn)和多節(jié)點(diǎn)兩個(gè)層面研究云環(huán)境下的能量高效的任務(wù)調(diào)度技術(shù)。本文的主要?jiǎng)?chuàng)新點(diǎn)如下:(1)由于現(xiàn)有的多核處理器節(jié)點(diǎn)上能量高效的任務(wù)調(diào)度方法通常假設(shè)可以獨(dú)立控制各個(gè)內(nèi)核的狀態(tài),在能耗計(jì)算階段只考慮處理器本身的能耗而未考慮節(jié)點(diǎn)其他部件的能耗,且對(duì)能效的度量方式未考慮實(shí)際意義,因此提出一種面向多種內(nèi)核結(jié)構(gòu)的代價(jià)感知的任務(wù)調(diào)度框架。綜合考慮處理器的靜態(tài)功耗、動(dòng)態(tài)功耗以及節(jié)點(diǎn)其他部件功耗,使用經(jīng)濟(jì)代價(jià)度量節(jié)點(diǎn)進(jìn)行任務(wù)處理的時(shí)間代價(jià)和能量代價(jià),將任務(wù)處理時(shí)間、等待時(shí)間和能量消耗代價(jià)統(tǒng)一。在此基礎(chǔ)上分別針對(duì)獨(dú)立控制、整體控制和分組控制三種內(nèi)核結(jié)構(gòu)的處理器設(shè)計(jì)不同的任務(wù)調(diào)度算法。使用該調(diào)度框架和任務(wù)調(diào)度算法可以降低節(jié)點(diǎn)的任務(wù)處理代價(jià):當(dāng)內(nèi)核為獨(dú)立控制結(jié)構(gòu)時(shí),負(fù)載越輕該方法相對(duì)傳統(tǒng)方法的優(yōu)勢(shì)越明顯;當(dāng)內(nèi)核為整體控制結(jié)構(gòu)時(shí),隨著負(fù)載加重該方法的節(jié)點(diǎn)代價(jià)低于傳統(tǒng)方法且二者之間的差距越來越大;當(dāng)內(nèi)核為分組控制結(jié)構(gòu)時(shí),該方法的節(jié)點(diǎn)代價(jià)與傳統(tǒng)方法相比成倍減少。(2)現(xiàn)有的云環(huán)境任務(wù)層能量高效的數(shù)據(jù)密集型任務(wù)調(diào)度方法主要通過改變數(shù)據(jù)存儲(chǔ)策略實(shí)現(xiàn)能量高效,此類方法與具體的數(shù)據(jù)存儲(chǔ)方法和存儲(chǔ)介質(zhì)相關(guān),不具備通用性,因此提出一種同構(gòu)節(jié)點(diǎn)環(huán)境下與數(shù)據(jù)存儲(chǔ)策略無關(guān)的能量高效的數(shù)據(jù)密集型任務(wù)處理方法EABD。在任務(wù)調(diào)度過程中綜合考慮處理任務(wù)的節(jié)點(diǎn)數(shù)和節(jié)點(diǎn)間的負(fù)載均衡情況,降低任務(wù)處理過程中的能量消耗。盡管該方法相對(duì)于傳統(tǒng)方法使用更多的節(jié)點(diǎn)處理任務(wù),但其任務(wù)處理過程中的能量消耗比傳統(tǒng)方法更小,在某些情況下其能量消耗甚至不足傳統(tǒng)方法的50%。該算法的能量消耗受副本數(shù)量影響較小,且在默認(rèn)3個(gè)副本的情況下該算法造成的能量浪費(fèi)最少。(3)針對(duì)實(shí)際云環(huán)境下的節(jié)點(diǎn)異構(gòu)性問題,提出一種異構(gòu)節(jié)點(diǎn)環(huán)境下與數(shù)據(jù)存儲(chǔ)策略和存儲(chǔ)介質(zhì)無關(guān)的能量高效的數(shù)據(jù)密集型任務(wù)處理方法MinBalance,將任務(wù)調(diào)度過程分為節(jié)點(diǎn)選擇和負(fù)載均衡兩步。在節(jié)點(diǎn)選擇過程中,定義四種不同的節(jié)點(diǎn)權(quán)值,根據(jù)貪心算法選擇權(quán)值最小的節(jié)點(diǎn)進(jìn)行任務(wù)處理。在負(fù)載均衡階段對(duì)參與任務(wù)處理的節(jié)點(diǎn)的負(fù)載進(jìn)行均衡,減少節(jié)點(diǎn)因等待而造成的能量浪費(fèi)。該方法充分考慮節(jié)點(diǎn)的性能和功耗的異構(gòu)性,降低任務(wù)處理的能量消耗,當(dāng)待處理的數(shù)據(jù)量較大時(shí),MinBalance可減少約60%的能量消耗。(4)針對(duì)目前能量高效的虛擬機(jī)調(diào)度方法主要考慮任務(wù)特征和資源分配,僅通過減少節(jié)點(diǎn)使用數(shù)量降低能耗的問題,提出一種云環(huán)境下能量高效的虛擬機(jī)調(diào)度算法EEVS。首先將虛擬機(jī)分配到擁有足夠資源且最優(yōu)性能功率比最高的物理機(jī)上執(zhí)行,從節(jié)點(diǎn)層減少能量消耗。在虛擬機(jī)執(zhí)行過程中采用基于DVFS的單節(jié)點(diǎn)節(jié)能技術(shù),通過虛擬機(jī)遷移進(jìn)行資源整合,減少每個(gè)物理機(jī)的能量消耗,從部件層進(jìn)一步降低系統(tǒng)的能量消耗。EEVS算法在不造成明顯效率降低的情況下,可以節(jié)約超過10%的能量消耗。(5)針對(duì)目前云計(jì)算應(yīng)用中僅考慮方法的可行性和效率提升,而不重視對(duì)能耗的優(yōu)化等問題,研究并分析了數(shù)據(jù)挖掘中常用的頻繁模式挖掘的能耗問題。根據(jù)本文提出的能量高效的任務(wù)調(diào)度方法,設(shè)計(jì)一種能量高效的任務(wù)調(diào)度器EEScheduler,對(duì)云環(huán)境下頻繁模式挖掘中的Map任務(wù)進(jìn)行能量高效的調(diào)度,從而降低系統(tǒng)的能量消耗。在四個(gè)節(jié)點(diǎn)組成的云平臺(tái)上進(jìn)行頻繁模式挖掘,實(shí)驗(yàn)結(jié)果表明EEScheduler可以降低60%以上的能量消耗。
[Abstract]:With the development and application of the information technology, the data generated in each industry is exploding. The traditional database technology can not solve the large-scale data application problem such as massive data, high concurrency, fast response, and expandability. Therefore, how to efficiently store and manage these data is a problem that needs to be solved at present. The economics, scalability and fault-tolerance of cloud computing make it the supporting technology of large data management. With the wide application of cloud computing, the number and size of the data center are growing rapidly, and the cost of the electricity bill has exceeded the purchase cost of the hardware equipment itself, and is still in the state of continuous growth. The rapid increase of energy consumption also exacerbates the global energy crisis and environmental pollution. So it is urgent to study the energy-efficient data management technology in the cloud environment. The energy efficient task scheduling technology in the cloud environment is an important part of the energy efficient data management technology in the cloud environment, and the aim of the invention is to reduce the energy consumption of the node used for task processing in the task processing process through the method of task scheduling. This paper studies the energy efficient task scheduling technology in the cloud environment from the single node and the multi-node. The main innovation points of this paper are as follows: (1) Since the energy efficient task scheduling method on the existing multi-core processor nodes generally assumes that the states of the respective cores can be controlled independently, only the energy consumption of the processor itself is taken into account in the energy consumption calculation stage, and the energy consumption of other components of the nodes is not taken into account, and therefore, a cost-aware task scheduling framework for a plurality of kernel structures is proposed. and comprehensively considering the static power consumption, the dynamic power consumption of the processor and the power consumption of other components of the node, and the time cost and the energy cost of the task processing are carried out by using the economic cost metric node, and the task processing time, the waiting time and the energy consumption cost are unified. On the basis of this, the different task scheduling algorithms are designed for the processor of the three kernel architectures, such as the independent control, the whole control and the packet control. using the scheduling framework and the task scheduling algorithm, the task processing cost of the nodes can be reduced; when the core is an independent control structure, the more the load is lighter; the more obvious the advantages of the method relative to the traditional method; when the core is an integral control structure, As the load increases the cost of the node of the method is lower than that of the traditional method and the gap between the two is larger and larger; when the core is a packet control structure, the node cost of the method is reduced by a plurality of times compared with the traditional method. and (2) the energy-efficient data-intensive task scheduling method of the existing cloud environment task layer mainly realizes the energy efficiency by changing the data storage strategy, and the method is related to the specific data storage method and the storage medium, does not have the universality, and therefore, an energy-efficient data-intensive task processing method, EABD, which is independent of the data storage strategy under the environment of a homogeneous node is proposed. in the task scheduling process, the node number of the processing task and the load balance among the nodes are comprehensively considered, and the energy consumption in the task processing process is reduced. Although the method uses more node processing tasks with respect to the conventional method, the energy consumption in its task processing is smaller than that of the conventional method, and in some cases its energy consumption is even less than 50% of the conventional method. The energy consumption of the algorithm is less affected by the number of copies, and the energy wasted by this algorithm is minimized in the case of a default of 3 copies. (3) A data-intensive task processing method, MinBalance, which is independent of the data storage strategy and the storage medium under the environment of a heterogeneous node is proposed, and the task scheduling process is divided into two steps: node selection and load balancing. in the node selection process, four different node weight values are defined, and the task processing is carried out according to the node with the smallest selection value of the greedy algorithm. the load balancing stage equalizes the load of the node participating in the task processing, and reduces the energy waste caused by the node waiting. The method fully considers the heterogeneity of the performance and power consumption of the node, reduces the energy consumption of the task processing, and when the amount of data to be processed is large, the MinBalance can reduce the energy consumption of about 60%. and (4) aiming at the problem that the current energy efficient virtual machine scheduling method mainly considers the task characteristics and the resource allocation, and only reduces the energy consumption by reducing the use quantity of the nodes, and proposes an energy efficient virtual machine scheduling algorithm EEVS under the cloud environment. first, the virtual machine is allocated to a physical machine that has sufficient resources and the optimal performance power is higher than the highest, and energy consumption is reduced from the node layer. During the execution of the virtual machine, the single-node energy-saving technology based on the DVFS is adopted, the resource integration is carried out through the migration of the virtual machine, the energy consumption of each physical machine is reduced, and the energy consumption of the system is further reduced from the component layer. The EEVS algorithm can save more than 10% of the energy consumption without causing significant efficiency degradation. (5) For the current cloud computing application, only the feasibility and efficiency of the method are considered, and the problem of energy consumption optimization and the like is not paid attention to, and the energy consumption problem of the frequent pattern mining used in data mining is analyzed and analyzed. According to the energy efficient task scheduling method proposed in this paper, an energy efficient task scheduler (EScheduler) is designed to efficiently and efficiently schedule the Map tasks in frequent pattern mining in the cloud environment, so as to reduce the energy consumption of the system. Frequent pattern mining is carried out on a cloud platform composed of four nodes, and the results show that the EEScheduler can reduce the energy consumption of more than 60%.
【學(xué)位授予單位】：南京航空航天大學(xué)
【學(xué)位級(jí)別】：博士
【學(xué)位授予年份】：2016
【分類號(hào)】：TP301.6

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 王立冬,張凱;Java虛擬機(jī)分析[J];北京理工大學(xué)學(xué)報(bào);2002年01期

2 曹曉剛;;Java虛擬機(jī)的10年[J];程序員;2005年07期

3 宋韜;盤細(xì)平;羅元柯;倪國軍;;Java虛擬機(jī)在嵌入式DSP系統(tǒng)上的實(shí)現(xiàn)[J];計(jì)算機(jī)應(yīng)用與軟件;2007年04期

4 劉黎波;;Java虛擬機(jī)攔截原理研究[J];科技風(fēng);2008年21期

5 劉治波;;Java虛擬機(jī)簡(jiǎn)析[J];濟(jì)南職業(yè)學(xué)院學(xué)報(bào);2008年01期

6 郝帥;;Java虛擬機(jī)中相關(guān)技術(shù)的探討[J];成功(教育);2008年08期

7 李霞;;系統(tǒng)虛擬機(jī)關(guān)鍵技術(shù)研究[J];微型電腦應(yīng)用;2010年03期

8 鄭曉瓏;孔挺;;虛擬機(jī)的安全風(fēng)險(xiǎn)與管理[J];硅谷;2010年16期

9 李學(xué)昌;平淡;;為速度而戰(zhàn),虛擬機(jī)內(nèi)外兼修[J];電腦愛好者;2010年18期

10 王惠萍;張海龍;馮帆;王建華;;Java虛擬機(jī)使用及優(yōu)化[J];計(jì)算機(jī)與網(wǎng)絡(luò);2010年21期

相關(guān)會(huì)議論文 前10條

1 孟廣平;;虛擬機(jī)漂移網(wǎng)絡(luò)連接方法探討[A];中國計(jì)量協(xié)會(huì)冶金分會(huì)2011年會(huì)論文集[C];2011年

2 段翼真;王曉程;;可信安全虛擬機(jī)平臺(tái)的研究[A];第26次全國計(jì)算機(jī)安全學(xué)術(shù)交流會(huì)論文集[C];2011年

3 李明宇;張倩;呂品;;網(wǎng)絡(luò)流量感知的虛擬機(jī)高可用動(dòng)態(tài)部署研究[A];2014第二屆中國指揮控制大會(huì)論文集（上）[C];2014年

4 林紅;;Java虛擬機(jī)面向數(shù)字媒體的應(yīng)用研究[A];計(jì)算機(jī)技術(shù)與應(yīng)用進(jìn)展——全國第17屆計(jì)算機(jī)科學(xué)與技術(shù)應(yīng)用（CACIS）學(xué)術(shù)會(huì)議論文集（上冊(cè)）[C];2006年

5 楊旭;彭一明;刑承杰;李若淼;;基于VMware vSphere 5虛擬機(jī)的備份系統(tǒng)實(shí)現(xiàn)[A];中國高等教育學(xué)會(huì)教育信息化分會(huì)第十二次學(xué)術(shù)年會(huì)論文集[C];2014年

6 沈敏虎;查德平;劉百祥;趙澤宇;;虛擬機(jī)網(wǎng)絡(luò)部署與管理研究[A];中國高等教育學(xué)會(huì)教育信息化分會(huì)第十次學(xué)術(shù)年會(huì)論文集[C];2010年

7 李英壯;廖培騰;孫夢(mèng);李先毅;;基于云計(jì)算的數(shù)據(jù)中心虛擬機(jī)管理平臺(tái)的設(shè)計(jì)[A];中國高等教育學(xué)會(huì)教育信息化分會(huì)第十次學(xué)術(shù)年會(huì)論文集[C];2010年

8 朱欣焰;蘇科華;毛繼國;龔健雅;;GIS符號(hào)虛擬機(jī)及實(shí)現(xiàn)方法研究[A];《測(cè)繪通報(bào)》測(cè)繪科學(xué)前沿技術(shù)論壇摘要集[C];2008年

9 于洋;陳曉東;俞承芳;李旦;;基于FPGA平臺(tái)的虛擬機(jī)建模與仿真[A];2007'儀表，自動(dòng)化及先進(jìn)集成技術(shù)大會(huì)論文集（一）[C];2007年

10 丁濤;郝沁汾;張冰;;內(nèi)核虛擬機(jī)調(diào)度策略的研究與分析[A];'2010系統(tǒng)仿真技術(shù)及其應(yīng)用學(xué)術(shù)會(huì)議論文集[C];2010年

相關(guān)重要報(bào)紙文章 前10條

1 ;虛擬機(jī)的生與死[N];網(wǎng)絡(luò)世界;2008年

2 本報(bào)記者 卜娜;高性能Java虛擬機(jī)將在中國云市場(chǎng)釋能[N];中國計(jì)算機(jī)報(bào);2012年

3 本報(bào)記者 邱燕娜;如何告別虛擬機(jī)管理煩惱[N];中國計(jì)算機(jī)報(bào);2012年

4 ;首批通過云計(jì)算產(chǎn)品虛擬機(jī)管理測(cè)評(píng)名單[N];中國電子報(bào);2014年

5 申琳;虛擬機(jī)泛濫 系統(tǒng)安全怎么辦[N];中國計(jì)算機(jī)報(bào);2008年

6 Tom Henderson邋沈建苗 編譯;虛擬機(jī)管理的五大問題[N];計(jì)算機(jī)世界;2008年

7 盆盆;真實(shí)的虛擬機(jī)[N];中國電腦教育報(bào);2004年

8 本版編輯　綜合 編譯整理 田夢(mèng);管理好虛擬機(jī)的全生命周期[N];計(jì)算機(jī)世界;2008年

9 李婷;中國研制出全球最快反病毒虛擬機(jī)[N];人民郵電;2009年

10 張弛;虛擬機(jī)遷移走向真正自由[N];網(wǎng)絡(luò)世界;2010年

相關(guān)博士學(xué)位論文 前10條

1 宋翔;多核虛擬環(huán)境的性能及可伸縮性研究[D];復(fù)旦大學(xué);2014年

2 王桂平;云環(huán)境下面向可信的虛擬機(jī)異常檢測(cè)關(guān)鍵技術(shù)研究[D];重慶大學(xué);2015年

3 周真;云平臺(tái)下運(yùn)行環(huán)境感知的虛擬機(jī)異常檢測(cè)策略及算法研究[D];重慶大學(xué);2015年

4 郭芬;面向虛擬機(jī)的云平臺(tái)資源部署與調(diào)度研究[D];華南理工大學(xué);2015年

5 周傲;高可靠云服務(wù)供應(yīng)關(guān)鍵技術(shù)研究[D];北京郵電大學(xué);2015年

6 劉圣卓;面向虛擬集群的鏡像存儲(chǔ)與傳輸優(yōu)化[D];清華大學(xué);2015年

7 彭成磊;云數(shù)據(jù)中心綠色節(jié)能需求的虛擬機(jī)負(fù)載均衡技術(shù)研究[D];南京大學(xué);2016年

8 趙長名;IaaS云中基于資源感知的虛擬機(jī)資源管埋[D];電子科技大學(xué);2016年

9 許小龍;支持綠色云計(jì)算的資源調(diào)度方法及關(guān)鍵技術(shù)研究[D];南京大學(xué);2016年

10 衷宜;虛擬化系統(tǒng)中的軟件自愈相關(guān)技術(shù)研究[D];南京理工大學(xué);2016年

相關(guān)碩士學(xué)位論文 前10條

1 潘飛;負(fù)載相關(guān)的虛擬機(jī)放置策略研究[D];杭州電子科技大學(xué);2011年

2 王建一;混合型桌面云高可用性研究與實(shí)現(xiàn)[D];華南理工大學(xué);2015年

3 周衡;云計(jì)算環(huán)境下虛擬機(jī)優(yōu)化調(diào)度策略研究[D];河北大學(xué);2015年

4 羅仲皓;基于OpenStack的私有云計(jì)算平臺(tái)的設(shè)計(jì)與實(shí)現(xiàn)[D];華南理工大學(xué);2015年

5 李子堂;面向負(fù)載均衡的虛擬機(jī)動(dòng)態(tài)遷移優(yōu)化研究[D];遼寧大學(xué);2015年

6 張煜;基于OpenStack的“實(shí)驗(yàn)云”平臺(tái)的研究與開發(fā)[D];西南交通大學(xué);2015年

7 曾文琦;面向應(yīng)用服務(wù)的云規(guī)模虛似機(jī)性能監(jiān)控與負(fù)載分析技術(shù)研究[D];復(fù)旦大學(xué);2013年

8 施繼成;面向多核處理器的虛擬機(jī)性能優(yōu)化[D];復(fù)旦大學(xué);2014年

9 游井輝;基于虛擬機(jī)動(dòng)態(tài)遷移的資源調(diào)度策略研究[D];華南理工大學(xué);2015年

10 方良英;云平臺(tái)的資源優(yōu)化管理研究與實(shí)現(xiàn)[D];南京師范大學(xué);2015年

，

本文編號(hào)：2426088