本文關(guān)鍵詞：虛擬計算集群下高性能應(yīng)用性能分析與優(yōu)化技術(shù)研究　出處：《國防科學(xué)技術(shù)大學(xué)》2013年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 虛擬化技術(shù) 虛擬計算集群 資源調(diào)度 科學(xué)工作流 Bin Packing

【摘要】：虛擬化帶來的性能開銷阻礙了高性能計算在虛擬集群環(huán)境下的大規(guī)模應(yīng)用。首先,虛擬化層次的引入在傳統(tǒng)操作系統(tǒng)任務(wù)調(diào)度中加入了虛擬CPU調(diào)度這一額外層次,帶來了額外的調(diào)度開銷。其次,同一臺物理主機(jī)上所有虛擬機(jī)的通信通過虛擬機(jī)監(jiān)視器統(tǒng)一處理,增加了應(yīng)用在虛擬集群環(huán)境下的通信延遲。最后,同一臺物理主機(jī)上的虛擬機(jī)對Cache等共享資源的非親和性使用,使得高性能應(yīng)用在虛擬集群環(huán)境下的性能顯著下降。分析高性能應(yīng)用在虛擬集群環(huán)境下的性能開銷并由此提出相應(yīng)的優(yōu)化技術(shù)是解決上述問題的有效手段。本文利用Xen虛擬化軟件、SLURM集群作業(yè)管理系統(tǒng),Eclipse+PTP遠(yuǎn)程并行開發(fā)調(diào)試環(huán)境等構(gòu)建了一個虛擬化高性能集群系統(tǒng),通過在該系統(tǒng)上執(zhí)行高性能測試集,對高性能應(yīng)用的虛擬化性能開銷進(jìn)行量化分析,提出了基于VCPU親和性的虛擬資源優(yōu)化調(diào)度技術(shù)。具體研究內(nèi)容包括以下三個方面:1.課題研究了虛擬化技術(shù)與基于虛擬集群的高性能計算國內(nèi)外現(xiàn)狀,為虛擬集群系統(tǒng)軟件棧的構(gòu)建提供了選擇依據(jù);對傳統(tǒng)高性能環(huán)境下的典型應(yīng)用進(jìn)行調(diào)研,選取了三個具有代表性的科學(xué)計算應(yīng)用作為虛擬化性能開銷的測試集;對用戶可能關(guān)心的性能指標(biāo)進(jìn)行了綜合,選取合適的性能評價標(biāo)準(zhǔn)衡量虛擬集群環(huán)境下高性能應(yīng)用的性能;最后闡述了虛擬集群系統(tǒng)的構(gòu)建流程。2.通過在虛擬集群和物理集群上分別執(zhí)行高性能應(yīng)用程序?qū)μ摂M化性能開銷進(jìn)行了量化分析。特別地,通過執(zhí)行中長期天氣預(yù)報模式WRF,對虛擬集群環(huán)境下實(shí)際科學(xué)應(yīng)用程序的性能開銷進(jìn)行了分析;通過執(zhí)行NPB標(biāo)準(zhǔn)測試集對虛擬集群下不同類型高性能應(yīng)用的性能開銷進(jìn)行了分析;通過執(zhí)行科學(xué)工作流數(shù)據(jù)放置策略測試了虛擬集群下工作流應(yīng)用不同規(guī)模通信量下的性能開銷。最后,基于量化分析結(jié)果,指出了基于虛擬集群的高性能計算當(dāng)前存在的問題與改進(jìn)方向。3.根據(jù)虛擬集群下通信復(fù)用不明顯而導(dǎo)致通信開銷較大的特點(diǎn),提出了基于Bin Packing的虛擬資源優(yōu)化調(diào)度算法。該算法在基于解決傳統(tǒng)Bin Packing問題的三種啟發(fā)式算法的基礎(chǔ)上,通過選取剩余時間和內(nèi)存作為權(quán)值估計函數(shù),設(shè)計了VM狀態(tài)轉(zhuǎn)換信息表。與先前算法相比,本文算法考慮了VCPU親和性對高性能應(yīng)用的性能影響,同時針對虛擬集群自身特點(diǎn)加入了通信感知與物理節(jié)點(diǎn)標(biāo)記機(jī)制,提出了考慮通信量的改進(jìn)Bin Packing算法。模擬實(shí)驗結(jié)果表明,基于Bin Packing的虛擬資源調(diào)度算法相對于比先前的資源調(diào)度算法在通信復(fù)用上具有較大優(yōu)勢。
[Abstract]:The performance overhead brought by virtualization hinders the large-scale application of high performance computing in virtual cluster environment. The introduction of virtualization layer adds virtual CPU scheduling to the traditional operating system task scheduling which brings additional scheduling overhead. Secondly. The communication of all virtual machines on the same physical host is processed uniformly by the virtual machine monitor, which increases the communication delay applied in the virtual cluster environment. Finally. The non-affinity usage of virtual machines on the same physical host to shared resources such as Cache. The performance of high performance applications in virtual cluster environment is significantly reduced. It is an effective method to solve the above problems by analyzing the performance overhead of high performance applications in virtual cluster environment and putting forward corresponding optimization techniques. Using Xen virtualization software. The SLURM Cluster Job Management system (SLURM), Eclipse PTP remote parallel development and debugging environment, has constructed a virtualized high performance cluster system, which executes the high performance test set on the system. The virtualization performance overhead of high performance applications is analyzed quantitatively. A virtual resource scheduling technology based on VCPU affinity is proposed. The specific research contents include the following three aspects: 1.The subject studies the virtualization technology and high performance computing based on virtual cluster at home and abroad. It provides the basis for choosing the software stack of virtual cluster system. This paper investigates the typical applications in the traditional high performance environment and selects three representative scientific computing applications as the test set of virtualization performance overhead. The performance indexes that the users may care about are synthesized, and the performance of high performance applications in virtual cluster environment is evaluated by selecting appropriate performance evaluation criteria. Finally, the construction process of virtual cluster system. 2. The virtualization performance overhead is analyzed quantitatively by executing high performance application programs on virtual cluster and physical cluster respectively. The performance overhead of actual scientific applications in virtual cluster environment is analyzed by implementing the medium and long term weather forecast model (WRFs). The performance overhead of different types of high performance applications in virtual cluster is analyzed by executing NPB standard test set. The performance overhead of workflow application under virtual cluster is tested by implementing scientific workflow data placement strategy. Finally, based on the results of quantitative analysis. This paper points out the existing problems and improvement direction of high performance computing based on virtual cluster. 3. According to the characteristics that communication reuse is not obvious in virtual cluster, which leads to high communication cost. A virtual resource scheduling algorithm based on Bin Packing is proposed, which is based on three heuristic algorithms to solve the traditional Bin Packing problem. By selecting the residual time and memory as the weight estimation function, the VM state transition information table is designed. Compared with the previous algorithm, this algorithm considers the effect of VCPU affinity on the performance of high performance applications. At the same time, according to the characteristics of virtual cluster, the communication perception and physical node marking mechanism are added, and an improved Bin Packing algorithm considering traffic is proposed. The virtual resource scheduling algorithm based on Bin Packing is superior to the previous resource scheduling algorithm in communication reuse.
【學(xué)位授予單位】：國防科學(xué)技術(shù)大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP302

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 劉杰;張亦添;遲利華;徐涵;蔣杰;胡慶豐;;并行作業(yè)容錯自動調(diào)度環(huán)境設(shè)計[J];計算機(jī)工程與科學(xué);2009年11期

，

本文編號：1400127