大規(guī)模復(fù)雜IT系統(tǒng)可靠性、性能、能耗關(guān)聯(lián)建模理論及其優(yōu)化技術(shù)研究
發(fā)布時間：2019-05-21 16:07
【摘要】：隨著互聯(lián)網(wǎng)的快速,以云計算和大數(shù)據(jù)處理技術(shù)為代表的新一代信息化技術(shù)不斷實現(xiàn)各類資源的整合和共享,以此形成了一種全新的大規(guī)模復(fù)雜IT系統(tǒng)(Large Scale Complex IT Systems,LSCITS)。相比傳統(tǒng)的IT系統(tǒng),其不僅需要有效地管理大規(guī)模、異構(gòu)復(fù)雜的基礎(chǔ)設(shè)施資源,同時也需要滿足多樣化的應(yīng)用需求,尤其是可靠計算、高性能計算和節(jié)能減排的應(yīng)用需求。為了實現(xiàn)大規(guī)模復(fù)雜IT系統(tǒng)下可靠、高效、節(jié)能的優(yōu)化調(diào)度管理,基于理論模型的系統(tǒng)指標評估必不可少,但是,在已有的相關(guān)研究中,可靠性、性能和能耗指標往往被視為相互分離的指標進行分析,而忽略了這些指標間相互影響的可靠性-性能-能耗(Reliability-Performance-Energy,R-P-E)關(guān)聯(lián)性;此外,大規(guī)模性的基礎(chǔ)設(shè)施資源也對面向多目標優(yōu)化的高效調(diào)度管理技術(shù)提出了新的挑戰(zhàn)。針對這些存在的關(guān)鍵性問題,本文對兩類典型的LSCITS(即云計算系統(tǒng)和大數(shù)據(jù)處理系統(tǒng))進行了系統(tǒng)全面的R-P-E關(guān)聯(lián)建模理論研究,同時將仿生自主神經(jīng)系統(tǒng)(Bionic Autonomic Nervous Systems,BANS)的思想用于調(diào)度管理系統(tǒng)的設(shè)計中,并基于建立的關(guān)聯(lián)模型進一步研究了可靠性、性能和能耗綜合考慮的優(yōu)化調(diào)度管理技術(shù)。論文的主要研究工作及創(chuàng)新性成果包括:1)提出了一種基于級層交互隨機子模型的建模方法(Hierarchical and Interacting Stochastic Models,HISM)。面向傳統(tǒng)服務(wù)系統(tǒng)遷移到云計算系統(tǒng)中的重要應(yīng)用場景,建立了相應(yīng)的R-P-E關(guān)聯(lián)模型。在基礎(chǔ)設(shè)施層首先建立了基于物理機和虛擬機失效修復(fù)行為的Semi-Markov可靠性模型,詳細分析了虛擬化環(huán)境下特有的物理機失效所引發(fā)的多虛擬機失效的復(fù)雜共因失效問題;其次,在應(yīng)用服務(wù)層,以可用資源量為條件參數(shù)建立了基于排隊論的性能模型,根據(jù)模型詳細分析了服務(wù)系統(tǒng)中重要的排隊溢出和超時失效等事件;在系統(tǒng)狀態(tài)監(jiān)測層,詳細分析了失效修復(fù)行為對系統(tǒng)動態(tài)能耗隨機變化帶來的影響,并建立了相應(yīng)的系統(tǒng)能耗模型;最后,通過馬爾可夫回報模型和貝葉斯理論提出了表征R-P-E關(guān)聯(lián)性的期望性能和期望能耗等綜合性評估指標,基于這些評估指標進一步提出了一種量化復(fù)雜P-E制約關(guān)系的新指標,即效能比(Performance-Energy Efficiency Ratio,PEER)。理論模型的分析結(jié)果通過仿真實驗進行了驗證,同時實驗結(jié)果表明效能比指標可有效幫助云計算系統(tǒng)為遷移的傳統(tǒng)服務(wù)系統(tǒng)選擇更加合理全面的資源分配策略。2)根據(jù)HISM建模方法,進一步基于新興的云服務(wù)系統(tǒng)(私有云服務(wù)系統(tǒng)和公有云服務(wù)系統(tǒng))建立了相應(yīng)的R-P-E關(guān)聯(lián)模型。針對多類型失效下的及時修復(fù)需求,提出了一種由多修復(fù)行為組成的級層修復(fù)機制,并建立了相應(yīng)的馬爾可夫可靠性模型。在私有云服務(wù)系統(tǒng)的性能分析方面,為了實現(xiàn)對核心云調(diào)度器運行狀態(tài)的分析,提出了一種新的Jackson排隊網(wǎng)絡(luò)模型,該模型不僅可以分析用戶請求在核心云調(diào)度器的請求解析時間,還可以分析虛擬機在資源池中的服務(wù)時間;在公有云服務(wù)系統(tǒng)的性能建模方面,更是充分考慮了用戶請求批量需求虛擬機的復(fù)雜行為特征。在云計算系統(tǒng)的能耗建模方面,不僅考慮了失效修復(fù)行帶來的隨機能耗變化情況,還考慮了服務(wù)用戶時隨機資源占用情況對系統(tǒng)動態(tài)能耗的影響。最后,通過仿真實驗驗證了云服務(wù)系統(tǒng)的R-P-E關(guān)聯(lián)模型,并詳細分析了資源分配決策變量影響下,云服務(wù)系統(tǒng)期望性能和期望能耗指標的重要變化趨勢。3)提出了另一種基于拉普拉斯變換(Laplace-Stieltjes Transform,LST)的關(guān)聯(lián)建模方法,建立了面向大數(shù)據(jù)處理系統(tǒng)的R-P-E關(guān)聯(lián)模型。在面向復(fù)雜計算任務(wù)時,針對任務(wù)完成時間直接影響實際能耗量的重要問題,提出了一種考慮了理想任務(wù)完成時間限制、硬件失效、數(shù)據(jù)處理程序失效等多種因素的Semi-Markov可靠性模型,并通過LST關(guān)聯(lián)建模方法實現(xiàn)了對期望任務(wù)執(zhí)行時間和期望能耗的分析評估。在面向大數(shù)據(jù)量任務(wù)時,充分考慮了子任務(wù)切分和子任務(wù)冗余執(zhí)行的復(fù)雜決策行為,并面向這種分布式冗余并行計算環(huán)境,設(shè)計了一種求解隨機任務(wù)完成時間概率分布函數(shù)的算法,最后基于貝葉斯理論建立了R-P-E關(guān)聯(lián)模型。實驗結(jié)果表明了所建立的理論模型對復(fù)雜計算任務(wù)最優(yōu)資源分配策略、大數(shù)據(jù)量任務(wù)最優(yōu)切分和冗余執(zhí)行策略的制定都有著重要的理論評估和分析作用。4)提出了基于R-P-E關(guān)聯(lián)模型的多目標優(yōu)化模型,并根據(jù)決策變量的類型和復(fù)雜性,設(shè)計了Pareto最優(yōu)解分析、收斂算法、遺傳算法等多種求解最優(yōu)解的方法,并建立一種基于仿生自主神經(jīng)系統(tǒng)(BANS)的新型云調(diào)度管理系統(tǒng)。在局部自主資源管理方面,基于用戶請求到達率敏感性分析的方法,建立了一種描述資源分配策略最優(yōu)性的“最優(yōu)性分布圖”,并進一步設(shè)計了基于最優(yōu)性分布圖的自主資源管理觸發(fā)機制,通過動態(tài)自主的資源再分配行為可以在用戶請求到達強度動態(tài)變化的環(huán)境下始終維持一種最優(yōu)的資源分配策略;在全局請求調(diào)度方面,設(shè)計了一種新的基于最優(yōu)性分布圖的優(yōu)化調(diào)度方法,從而避免了核心云調(diào)度節(jié)點對大規(guī)模復(fù)雜的基礎(chǔ)設(shè)施資源進行繁瑣的最優(yōu)解搜索。實驗結(jié)果顯示基于BANS的云調(diào)度管理系統(tǒng)可以在系統(tǒng)期望純利潤上取得良好的優(yōu)化效果,同時還有效提升了核心云調(diào)度節(jié)點搜索最優(yōu)解的效率。
[Abstract]:With the rapid development of the Internet, the new generation of information technology, which is represented by cloud computing and big data processing technology, continuously realizes the integration and sharing of all kinds of resources, thus forming a brand-new large-scale complex IT system (LSCITS). Compared with the traditional IT system, it not only needs to effectively manage large-scale, heterogeneous and complex infrastructure resources, but also needs to meet the diversified application requirements, especially the application requirements of reliable calculation, high performance calculation and energy-saving and emission reduction. In order to realize the reliable, efficient and energy-saving optimal scheduling management under the large-scale complex IT system, the system index evaluation based on the theoretical model is essential, but in the existing research, the reliability, the performance and the energy consumption index are often regarded as the mutually separated indexes for analysis, The reliability-performance-energy (R-P-E) relevance of the interaction among these indicators is ignored; in addition, the large-scale infrastructure resource also presents new challenges to the high-efficiency scheduling management technology facing the multi-objective optimization. In this paper, two typical LSCITS, namely the cloud computing system and the large data processing system, are studied in this paper. The idea of BANS is used in the design of the scheduling management system, and the optimal scheduling management technology for reliability, performance and energy consumption is further studied based on the established association model. The main research work and innovative achievements of the paper include:1) a modeling method based on a hierarchical layer interaction random sub-model (HISM) is proposed. The corresponding R-P-E association model is established for the application of the traditional service system to the cloud computing system. firstly, a semi-Markov reliability model based on the failure repair behavior of the physical machine and the virtual machine is established on the infrastructure layer, the problem of the complex common cause failure of the multi-virtual machine failure caused by the failure of the special physical machine under the virtualization environment is analyzed in detail, and secondly, in the application service layer, The performance model based on the queuing theory is established with the available resources as the condition parameters, and the events such as queuing overflow and timeout failure in the service system are analyzed in detail according to the model, and the system state monitoring layer, In this paper, the influence of the failure repair behavior on the random change of the dynamic energy consumption of the system is analyzed in detail, and the corresponding system energy consumption model is established; and finally, a comprehensive evaluation index, such as the expected performance and the expected energy consumption of the R-P-E association, is proposed through the Markov model and the Bayesian theory. Based on these evaluation indexes, a new index, i.e., performance-energy efficiency Ratio (PEER), is proposed to quantify the complex P-E constraints. The results show that the performance ratio index can help the cloud computing system to choose a more reasonable and comprehensive resource allocation strategy for the traditional service system. The corresponding R-P-E association model is further developed based on the new cloud service system (private cloud service system and public cloud service system). In view of the need for timely repair in multi-type failure, a level-level repair mechanism composed of multiple repair behaviors is proposed, and a corresponding Markov reliability model is established. In order to realize the analysis of the operation state of the core cloud scheduler in the performance analysis of the private cloud service system, a new Jackson queuing network model is proposed, which can not only analyze the request resolution time of the user request at the core cloud scheduler, The service time of the virtual machine in the resource pool can also be analyzed; in the aspect of the performance modeling of the public cloud service system, the complex behavior characteristics of the user request batch demand virtual machine are fully taken into account. In the aspect of energy consumption modeling of the cloud computing system, not only the random energy consumption change caused by the failure repair line is considered, but also the influence of the random resource occupation situation on the dynamic energy consumption of the system is also taken into account. Finally, the R-P-E association model of the cloud service system is verified by the simulation experiment, and the important variation trend of the expected performance and the expected energy consumption index of the cloud service system under the influence of the resource allocation decision variables is analyzed in detail. In this paper, an R-P-E association model for a large data processing system is established based on the association modeling method of LST. In view of the important problem of the task completion time directly affecting the actual energy consumption, a semi-Markov reliability model, which takes into account various factors such as the ideal task completion time limit, the hardware failure, the data processing program failure, and the like, is proposed in the face of the complex calculation task. And the analysis and evaluation of the expected task execution time and the expected energy consumption are realized through the LST association modeling method. In that task of large data volume, the complex decision behavior of sub-task segmentation and sub-task redundancy is fully taken into account, and the algorithm for solving the probability distribution function of the time probability distribution of the random task is designed for the distributed redundant parallel computing environment. Finally, the R-P-E correlation model is established based on the Bayesian theory. The experimental results show that the established theory model has important theoretical evaluation and analysis function for the optimal resource allocation strategy of complex computing task, the optimal segmentation of the large data volume task and the development of the redundant execution strategy.4) The multi-objective optimization model based on the R-P-E correlation model is proposed. Based on the type and complexity of the decision variable, a new method for solving the optimal solution, such as Pareto optimal solution analysis, convergence algorithm and genetic algorithm, is designed, and a new type of cloud scheduling management system based on the bionic autonomous nervous system (BANS) is established. In the aspect of local independent resource management, based on the method of user request arrival rate sensitivity analysis, a "optimality profile" describing the optimality of resource allocation strategy is set up, and an autonomous resource management trigger mechanism based on the optimality profile is further designed, the dynamic independent resource reallocation behavior can always maintain an optimal resource allocation strategy in the environment where the user requests to reach the dynamic change of the intensity; in the aspect of global request scheduling, a new optimal scheduling method based on the optimality profile is designed, Thereby avoiding the complex optimal solution search of the core cloud scheduling node to the large-scale complex infrastructure resource. The results show that the cloud scheduling management system based on the BANS can achieve a good optimization effect on the expected net profit of the system, and also effectively improves the efficiency of searching the optimal solution for the core cloud scheduling node.
【學位授予單位】：電子科技大學
【學位級別】：博士
【學位授予年份】：2016
【分類號】：TP302.7
[Abstract]:With the rapid development of the Internet, the new generation of information technology, which is represented by cloud computing and big data processing technology, continuously realizes the integration and sharing of all kinds of resources, thus forming a brand-new large-scale complex IT system (LSCITS). Compared with the traditional IT system, it not only needs to effectively manage large-scale, heterogeneous and complex infrastructure resources, but also needs to meet the diversified application requirements, especially the application requirements of reliable calculation, high performance calculation and energy-saving and emission reduction. In order to realize the reliable, efficient and energy-saving optimal scheduling management under the large-scale complex IT system, the system index evaluation based on the theoretical model is essential, but in the existing research, the reliability, the performance and the energy consumption index are often regarded as the mutually separated indexes for analysis, The reliability-performance-energy (R-P-E) relevance of the interaction among these indicators is ignored; in addition, the large-scale infrastructure resource also presents new challenges to the high-efficiency scheduling management technology facing the multi-objective optimization. In this paper, two typical LSCITS, namely the cloud computing system and the large data processing system, are studied in this paper. The idea of BANS is used in the design of the scheduling management system, and the optimal scheduling management technology for reliability, performance and energy consumption is further studied based on the established association model. The main research work and innovative achievements of the paper include:1) a modeling method based on a hierarchical layer interaction random sub-model (HISM) is proposed. The corresponding R-P-E association model is established for the application of the traditional service system to the cloud computing system. firstly, a semi-Markov reliability model based on the failure repair behavior of the physical machine and the virtual machine is established on the infrastructure layer, the problem of the complex common cause failure of the multi-virtual machine failure caused by the failure of the special physical machine under the virtualization environment is analyzed in detail, and secondly, in the application service layer, The performance model based on the queuing theory is established with the available resources as the condition parameters, and the events such as queuing overflow and timeout failure in the service system are analyzed in detail according to the model, and the system state monitoring layer, In this paper, the influence of the failure repair behavior on the random change of the dynamic energy consumption of the system is analyzed in detail, and the corresponding system energy consumption model is established; and finally, a comprehensive evaluation index, such as the expected performance and the expected energy consumption of the R-P-E association, is proposed through the Markov model and the Bayesian theory. Based on these evaluation indexes, a new index, i.e., performance-energy efficiency Ratio (PEER), is proposed to quantify the complex P-E constraints. The results show that the performance ratio index can help the cloud computing system to choose a more reasonable and comprehensive resource allocation strategy for the traditional service system. The corresponding R-P-E association model is further developed based on the new cloud service system (private cloud service system and public cloud service system). In view of the need for timely repair in multi-type failure, a level-level repair mechanism composed of multiple repair behaviors is proposed, and a corresponding Markov reliability model is established. In order to realize the analysis of the operation state of the core cloud scheduler in the performance analysis of the private cloud service system, a new Jackson queuing network model is proposed, which can not only analyze the request resolution time of the user request at the core cloud scheduler, The service time of the virtual machine in the resource pool can also be analyzed; in the aspect of the performance modeling of the public cloud service system, the complex behavior characteristics of the user request batch demand virtual machine are fully taken into account. In the aspect of energy consumption modeling of the cloud computing system, not only the random energy consumption change caused by the failure repair line is considered, but also the influence of the random resource occupation situation on the dynamic energy consumption of the system is also taken into account. Finally, the R-P-E association model of the cloud service system is verified by the simulation experiment, and the important variation trend of the expected performance and the expected energy consumption index of the cloud service system under the influence of the resource allocation decision variables is analyzed in detail. In this paper, an R-P-E association model for a large data processing system is established based on the association modeling method of LST. In view of the important problem of the task completion time directly affecting the actual energy consumption, a semi-Markov reliability model, which takes into account various factors such as the ideal task completion time limit, the hardware failure, the data processing program failure, and the like, is proposed in the face of the complex calculation task. And the analysis and evaluation of the expected task execution time and the expected energy consumption are realized through the LST association modeling method. In that task of large data volume, the complex decision behavior of sub-task segmentation and sub-task redundancy is fully taken into account, and the algorithm for solving the probability distribution function of the time probability distribution of the random task is designed for the distributed redundant parallel computing environment. Finally, the R-P-E correlation model is established based on the Bayesian theory. The experimental results show that the established theory model has important theoretical evaluation and analysis function for the optimal resource allocation strategy of complex computing task, the optimal segmentation of the large data volume task and the development of the redundant execution strategy.4) The multi-objective optimization model based on the R-P-E correlation model is proposed. Based on the type and complexity of the decision variable, a new method for solving the optimal solution, such as Pareto optimal solution analysis, convergence algorithm and genetic algorithm, is designed, and a new type of cloud scheduling management system based on the bionic autonomous nervous system (BANS) is established. In the aspect of local independent resource management, based on the method of user request arrival rate sensitivity analysis, a "optimality profile" describing the optimality of resource allocation strategy is set up, and an autonomous resource management trigger mechanism based on the optimality profile is further designed, the dynamic independent resource reallocation behavior can always maintain an optimal resource allocation strategy in the environment where the user requests to reach the dynamic change of the intensity; in the aspect of global request scheduling, a new optimal scheduling method based on the optimality profile is designed, Thereby avoiding the complex optimal solution search of the core cloud scheduling node to the large-scale complex infrastructure resource. The results show that the cloud scheduling management system based on the BANS can achieve a good optimization effect on the expected net profit of the system, and also effectively improves the efficiency of searching the optimal solution for the core cloud scheduling node.
【學位授予單位】：電子科技大學
【學位級別】：博士
【學位授予年份】：2016
【分類號】：TP302.7
【相似文獻】
相關(guān)期刊論文 前4條
1 李光明;蔡福俊;張慧琳;;基于灰色等維新息模型的能耗預(yù)測研究及應(yīng)用[J];微型機與應(yīng)用;2009年22期
2 ;綠色“芯”動力——性能VS能耗 魚與熊掌也可兼得[J];信息方略;2008年12期
3 周，
本文編號：2482216
本文鏈接：http://sikaile.net/shoufeilunwen/xxkjbs/2482216.html
最近更新
教材專著
