虛擬化集群的磁盤IO QoS控制
發(fā)布時(shí)間：2018-08-31 10:24
【摘要】：在計(jì)算機(jī)科學(xué)領(lǐng)域，云計(jì)算目前被視為客戶機(jī)/服務(wù)器架構(gòu)變革后的另一大技術(shù)革新。為了滿足云用戶的不同服務(wù)需求，云計(jì)算系統(tǒng)必須統(tǒng)一管理一個(gè)巨大的資源集群。因此，云資源管理成為當(dāng)前研究人員所面臨的一個(gè)巨大挑戰(zhàn)。云計(jì)算資源管理的主要任務(wù)包括數(shù)據(jù)存儲(chǔ)和資源管理兩方面，二者都涉及到磁盤IO資源。磁盤IO資源是云計(jì)算環(huán)境下所有資源中重要的一個(gè)，一個(gè)高效的磁盤IO QoS系統(tǒng)將會(huì)為云計(jì)算服務(wù)供應(yīng)商、云用戶和終端用戶帶來(lái)很大的便利。然而，在設(shè)計(jì)和實(shí)現(xiàn)一個(gè)磁盤IO QoS調(diào)度系統(tǒng)時(shí)需要考慮許多因素，諸如系統(tǒng)高性能、管理靈活性、兼容性等等。如何實(shí)現(xiàn)虛擬化存儲(chǔ)的磁盤IO資源調(diào)度策略并且確保磁盤IO的服務(wù)質(zhì)量成為具有實(shí)踐意義的重要研究課題。 本論文的研究工作來(lái)源于國(guó)家863計(jì)劃課題項(xiàng)目。該項(xiàng)目期望能夠通過(guò)現(xiàn)有的虛擬化技術(shù)和流量控制技術(shù)實(shí)現(xiàn)一種有效的磁盤流量的控制方案，，以實(shí)現(xiàn)對(duì)特定單元進(jìn)行流量控制，保證系統(tǒng)資源的高可用性，確保云計(jì)算服務(wù)提供商為用戶提供優(yōu)質(zhì)、穩(wěn)定的數(shù)據(jù)流量方案。針對(duì)以上需求，本文提出一個(gè)基于虛擬化技術(shù)和QoS控制技術(shù)的磁盤IO資源管理方案，對(duì)磁盤IO流量進(jìn)行有效管理和控制。這一磁盤IO管理方案具有提供多種服務(wù)質(zhì)量的高性能和管理的靈活性，并在已有的系統(tǒng)測(cè)試環(huán)境下對(duì)模塊進(jìn)行測(cè)試優(yōu)化，為線上運(yùn)行的虛擬化集群環(huán)境提供高可靠保證的實(shí)驗(yàn)驗(yàn)證。 在本論文中，具體研究工作及創(chuàng)新包括以下幾方面： 1.設(shè)計(jì)和實(shí)現(xiàn)磁盤IO QoS控制模塊。利用漏桶算法和加權(quán)公平隊(duì)列算法的結(jié)合模型，實(shí)現(xiàn)兩種對(duì)應(yīng)的控制策略，用于限制系統(tǒng)資源的throttle策略和用于針對(duì)不同請(qǐng)求分配不同優(yōu)先級(jí)的weight策略。利用device mapper技術(shù)將模塊實(shí)現(xiàn)為獨(dú)立的虛擬化塊設(shè)備層，降低引入模塊對(duì)系統(tǒng)性能的影響。 2.實(shí)現(xiàn)QoS管理控制接口。修改虛擬化平臺(tái)的相關(guān)接口腳本，加入QoS相關(guān)設(shè)計(jì)，實(shí)現(xiàn)虛擬設(shè)備QoS參數(shù)的動(dòng)態(tài)配置，方便系統(tǒng)整合和參數(shù)管理。模塊整合后，評(píng)估并分析其對(duì)現(xiàn)有系統(tǒng)的影響。 3.分析磁盤IO優(yōu)化因素。對(duì)系統(tǒng)磁盤IO進(jìn)行測(cè)試分析，結(jié)果表明，通過(guò)加入對(duì)IO請(qǐng)求大小和并發(fā)IO數(shù)量等因素的控制可以提高磁盤IO效率，優(yōu)化系統(tǒng)磁盤性能。 本論文通過(guò)對(duì)虛擬化環(huán)境下的磁盤資源管理技術(shù)現(xiàn)狀進(jìn)行分析和研究，提出基于虛擬化技術(shù)的磁盤IO QoS控制模塊，選擇合理的設(shè)計(jì)方案并利用device mapper技術(shù)進(jìn)行實(shí)現(xiàn)。為了使該模塊更易于系統(tǒng)管理，文中也提供了磁盤QoS管理的設(shè)計(jì)方案和實(shí)現(xiàn)細(xì)節(jié)。經(jīng)驗(yàn)證，該模塊具備完整的磁盤IO QoS功能，能夠提供多種IO調(diào)度策略，為用戶提供不同的服務(wù)質(zhì)量，確保系統(tǒng)效率。其策略能夠限制磁盤IO資源在某一值附近，波動(dòng)不超過(guò)整體的3%，防止突發(fā)流量的情況，并且在滿足功能需求的基礎(chǔ)上，將模塊整合帶給系統(tǒng)的影響控制在2%以內(nèi)。
[Abstract]:In the field of computer science, cloud computing is now seen as another major technological innovation following the change in client / server architecture. In order to meet the different needs of cloud users, cloud computing systems must manage a huge cluster of resources. Therefore, cloud resource management has become a great challenge for researchers. The main tasks of cloud computing resource management include data storage and resource management, both of which involve disk IO resources. Disk IO resource is one of the most important resources in cloud computing environment. An efficient disk IO QoS system will bring great convenience to cloud computing service providers, cloud users and end users. However, many factors need to be taken into account when designing and implementing a disk IO QoS scheduling system, such as system high performance, management flexibility, compatibility and so on. How to implement the disk IO resource scheduling strategy of virtualized storage and ensure the quality of service of disk IO has become an important research topic with practical significance. The research work of this thesis comes from the national 863 project. The project hopes to realize an effective disk traffic control scheme through the existing virtualization technology and flow control technology, in order to achieve flow control of specific units and ensure the high availability of system resources. Ensure that cloud computing service providers provide users with quality, stable data flow solutions. According to the above requirements, this paper proposes a disk IO resource management scheme based on virtualization technology and QoS control technology, which can effectively manage and control disk IO traffic. The disk IO management scheme has the advantages of high performance and flexibility to provide multiple quality of service, and the modules are tested and optimized under the existing system test environment. It provides high reliable experimental verification for the virtualized cluster environment running on the line. In this paper, the specific research work and innovation includes the following aspects: 1. Design and implement disk IO QoS control module. Using the combination model of leaky bucket algorithm and weighted fair queue algorithm, two corresponding control strategies are implemented, which are used to restrict the system resource throttle policy and to assign different priority weight policy to different requests. The module is realized as an independent virtualized block device layer by using device mapper technology to reduce the impact of the introduced module on the system performance. 2. Implement QoS management control interface. Modify the interface script of virtualization platform, add QoS related design, realize the dynamic configuration of QoS parameters of virtual device, facilitate system integration and parameter management. After module integration, evaluate and analyze its impact on existing systems. Analyze disk IO optimization factors. The system disk IO is tested and analyzed. The results show that the system disk performance can be optimized by adding the control factors such as the size of IO request and the number of concurrent IO to improve the disk IO efficiency. Based on the analysis and research on the current situation of disk resource management technology in virtualization environment, this paper proposes a disk IO QoS control module based on virtualization technology, selects a reasonable design scheme and implements it by using device mapper technology. In order to make the module easier to manage, the paper also provides the design and implementation details of disk QoS management. It has been proved that the module has complete disk IO QoS function and can provide a variety of IO scheduling strategies to provide users with different quality of service and ensure the efficiency of the system. The strategy can limit disk IO resources near a certain value, fluctuate less than 3 percent of the whole, prevent sudden traffic, and control the impact of module integration to less than 2 percent on the basis of meeting the functional requirements.
【學(xué)位授予單位】：上海交通大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333.35
本文編號(hào)：2214704
[Abstract]:In the field of computer science, cloud computing is now seen as another major technological innovation following the change in client / server architecture. In order to meet the different needs of cloud users, cloud computing systems must manage a huge cluster of resources. Therefore, cloud resource management has become a great challenge for researchers. The main tasks of cloud computing resource management include data storage and resource management, both of which involve disk IO resources. Disk IO resource is one of the most important resources in cloud computing environment. An efficient disk IO QoS system will bring great convenience to cloud computing service providers, cloud users and end users. However, many factors need to be taken into account when designing and implementing a disk IO QoS scheduling system, such as system high performance, management flexibility, compatibility and so on. How to implement the disk IO resource scheduling strategy of virtualized storage and ensure the quality of service of disk IO has become an important research topic with practical significance. The research work of this thesis comes from the national 863 project. The project hopes to realize an effective disk traffic control scheme through the existing virtualization technology and flow control technology, in order to achieve flow control of specific units and ensure the high availability of system resources. Ensure that cloud computing service providers provide users with quality, stable data flow solutions. According to the above requirements, this paper proposes a disk IO resource management scheme based on virtualization technology and QoS control technology, which can effectively manage and control disk IO traffic. The disk IO management scheme has the advantages of high performance and flexibility to provide multiple quality of service, and the modules are tested and optimized under the existing system test environment. It provides high reliable experimental verification for the virtualized cluster environment running on the line. In this paper, the specific research work and innovation includes the following aspects: 1. Design and implement disk IO QoS control module. Using the combination model of leaky bucket algorithm and weighted fair queue algorithm, two corresponding control strategies are implemented, which are used to restrict the system resource throttle policy and to assign different priority weight policy to different requests. The module is realized as an independent virtualized block device layer by using device mapper technology to reduce the impact of the introduced module on the system performance. 2. Implement QoS management control interface. Modify the interface script of virtualization platform, add QoS related design, realize the dynamic configuration of QoS parameters of virtual device, facilitate system integration and parameter management. After module integration, evaluate and analyze its impact on existing systems. Analyze disk IO optimization factors. The system disk IO is tested and analyzed. The results show that the system disk performance can be optimized by adding the control factors such as the size of IO request and the number of concurrent IO to improve the disk IO efficiency. Based on the analysis and research on the current situation of disk resource management technology in virtualization environment, this paper proposes a disk IO QoS control module based on virtualization technology, selects a reasonable design scheme and implements it by using device mapper technology. In order to make the module easier to manage, the paper also provides the design and implementation details of disk QoS management. It has been proved that the module has complete disk IO QoS function and can provide a variety of IO scheduling strategies to provide users with different quality of service and ensure the efficiency of the system. The strategy can limit disk IO resources near a certain value, fluctuate less than 3 percent of the whole, prevent sudden traffic, and control the impact of module integration to less than 2 percent on the basis of meeting the functional requirements.
【學(xué)位授予單位】：上海交通大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333.35
【參考文獻(xiàn)】
相關(guān)期刊論文 前6條
1 晉曉輝,李建東;WFQ的性能分析及改進(jìn)[J];高技術(shù)通訊;2002年07期
2 張魯靖,于建云;Fuzzy令牌桶模型在數(shù)據(jù)管理系統(tǒng)中的應(yīng)用[J];計(jì)算機(jī)工程與應(yīng)用;2004年04期
3 戴世冬;段海新;李星;;基于令牌桶陣列的DDoS流量過(guò)濾[J];清華大學(xué)學(xué)報(bào)(自然科學(xué)版);2011年01期
4 姚玉坤;王智;;改善NGN中多業(yè)務(wù)QoS的令牌桶與WFQ調(diào)度策略分析[J];信息技術(shù)與信息化;2006年05期
5 鄒永貴;萬(wàn)建斌;;云計(jì)算環(huán)境下的資源管理研究[J];數(shù)字通信;2012年04期
6 許玲;基于漏桶+WFQ的QoS調(diào)度策略分析[J];微機(jī)發(fā)展;2003年08期
本文編號(hào)：2214704
本文鏈接：http://sikaile.net/kejilunwen/jisuanjikexuelunwen/2214704.html
最近更新
教材專著
