基于HDFS的虛擬機(jī)鏡像存儲機(jī)制研究
發(fā)布時間：2019-03-01 17:44
【摘要】：隨著信息技術(shù)的迅速發(fā)展及互聯(lián)網(wǎng)數(shù)據(jù)的急劇膨脹,互聯(lián)網(wǎng)應(yīng)用的動態(tài)資源需求快速增長,增加了服務(wù)器的投資需求,給互聯(lián)網(wǎng)企業(yè)帶來了巨大的經(jīng)濟(jì)負(fù)擔(dān)。然而,服務(wù)器增加易造成資源浪費,特別在系統(tǒng)負(fù)載較低時,大量的服務(wù)器將處于閑置狀態(tài)。云計算的資源高效利用特性為企業(yè)降低運營成本、資源的按需分配和自動部署提供了可行途徑。作為云計算的核心支撐技術(shù),虛擬化能對服務(wù)器進(jìn)行有效整合,支持虛擬機(jī)動態(tài)遷移,實現(xiàn)服務(wù)器的動態(tài)封裝和運行時的負(fù)載均衡。近年來,多種虛擬化產(chǎn)品(如Xen, KVM)被開發(fā)出來,有效支撐了云計算的發(fā)展。 在典型的云計算環(huán)境中,虛擬機(jī)的部署需要高效存儲設(shè)備的支持,而目前Xen等虛擬化方式采用NAS、SAN等作為存儲模式,難以滿足實際應(yīng)用需要：一方面高速存儲設(shè)備價格高昂、設(shè)備能耗較高且存儲空間利用率較低；另一方面單一設(shè)備的低容錯率易造成數(shù)據(jù)丟失或錯誤。針對云計算虛擬機(jī)環(huán)境中的存儲問題,業(yè)界也提出了多種云存儲方案,通過將眾多異構(gòu)、廉價的服務(wù)器和存儲設(shè)備集合起來協(xié)同工作,提供數(shù)據(jù)存儲和訪問服務(wù)。其中的Hadoop分布式文件系統(tǒng)HDFS實現(xiàn)了海量數(shù)據(jù)的存儲管理,并具有高可用性、高可靠性、高穩(wěn)定性和低成本等優(yōu)勢,在許多企業(yè)和科研機(jī)構(gòu)中得到了廣泛應(yīng)用。 但是現(xiàn)有的HDFS實現(xiàn)也存在數(shù)據(jù)訪問延遲較大、不支持隨機(jī)寫等問題,限制了對虛擬機(jī)存儲支持的應(yīng)用。本文針對云計算環(huán)境下的Xen虛擬機(jī)存儲問題,基于HDFS設(shè)計并實現(xiàn)了一個虛擬化平臺解決方案,將虛擬機(jī)鏡像存儲擴(kuò)展到云存儲平臺HDFS。論文主要工作包括以下幾個方面： (1)深入分析了虛擬機(jī)動態(tài)遷移過程的性能瓶頸問題,提出了一種基于HDFS體系結(jié)構(gòu)的虛擬機(jī)鏡像存儲框架,建立了支持虛擬機(jī)鏡像動態(tài)運行的分布容錯云存儲機(jī)制。 (2)以HDFS系統(tǒng)為基礎(chǔ),通過引入日志結(jié)構(gòu)文件系統(tǒng)(LFS)的思想,設(shè)計了一種支持虛擬機(jī)鏡像存儲的塊級存儲系統(tǒng)HBISS,解決了IDFS無法支持隨機(jī)寫的缺陷,為鏡像數(shù)據(jù)的動態(tài)訪問提供了支持。 (3)結(jié)合Xen虛擬機(jī)訪問鏡像文件的特性,通過對Xen內(nèi)帶的blktap工具擴(kuò)展,建立了從虛擬機(jī)I/O到分布式云存儲文件的訪問機(jī)制,實現(xiàn)了虛擬機(jī)對HBISS鏡像的訪問。
[Abstract]:With the rapid development of information technology and the rapid expansion of Internet data, the dynamic resource demand for Internet applications is growing rapidly, which increases the investment demand of servers and brings a huge economic burden to Internet enterprises. However, the increase of servers is easy to waste resources, especially when the system load is low, a large number of servers will be idle. The efficient resource utilization of cloud computing provides a feasible way for enterprises to reduce operating costs, allocate resources on demand and deploy them automatically. As the core supporting technology of cloud computing, virtualization can effectively integrate servers, support dynamic migration of virtual machines, and realize dynamic encapsulation of servers and load balancing at runtime. In recent years, a variety of virtualization products (such as Xen, KVM) have been developed to effectively support the development of cloud computing. In a typical cloud computing environment, the deployment of virtual machines needs the support of efficient storage devices, but at present, Xen and other virtualization methods adopt NAS,SAN and other storage modes, which are difficult to meet the needs of practical applications: on the one hand, high-speed storage devices are expensive, The energy consumption of the equipment is high and the utilization of storage space is low. On the other hand, the low fault tolerance rate of a single device is easy to cause data loss or error. In order to solve the storage problem in cloud computing virtual machine environment, many cloud storage schemes are proposed, which can provide data storage and access services by integrating many heterogeneous and cheap servers and storage devices together to provide data storage and access services. The Hadoop distributed file system (HDFS), which implements the storage management of massive data, has the advantages of high availability, high reliability, high stability and low cost, and has been widely used in many enterprises and scientific research institutions. However, the existing implementation of HDFS also has some problems such as large delay of data access, not supporting random writing and so on, which limits the application of storage support for virtual machines. Aiming at the storage problem of Xen virtual machine in cloud computing environment, this paper designs and implements a virtual platform solution based on HDFS, which extends the virtual machine mirror storage to the cloud storage platform HDFS.. The main work of this paper is as follows: (1) the performance bottleneck of virtual machine dynamic migration is analyzed deeply, and a virtual machine mirror storage framework based on HDFS architecture is proposed. A distributed fault-tolerant cloud storage mechanism which supports the dynamic running of virtual machine mirror is established. (2) based on HDFS system, a block-level storage system HBISS, which supports virtual machine mirror storage is designed by introducing the idea of log structure file system (LFS), which solves the defect that IDFS can't support random writing. Provides support for dynamic access to mirrored data. (3) according to the characteristic of Xen virtual machine accessing image file, the access mechanism from virtual machine I / o to distributed cloud storage file is established by extending the blktap tool in Xen, and the access mechanism of virtual machine to HBISS image is realized.
【學(xué)位授予單位】：南京大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2012
【分類號】：TP302;TP333
本文編號：2432658
[Abstract]:With the rapid development of information technology and the rapid expansion of Internet data, the dynamic resource demand for Internet applications is growing rapidly, which increases the investment demand of servers and brings a huge economic burden to Internet enterprises. However, the increase of servers is easy to waste resources, especially when the system load is low, a large number of servers will be idle. The efficient resource utilization of cloud computing provides a feasible way for enterprises to reduce operating costs, allocate resources on demand and deploy them automatically. As the core supporting technology of cloud computing, virtualization can effectively integrate servers, support dynamic migration of virtual machines, and realize dynamic encapsulation of servers and load balancing at runtime. In recent years, a variety of virtualization products (such as Xen, KVM) have been developed to effectively support the development of cloud computing. In a typical cloud computing environment, the deployment of virtual machines needs the support of efficient storage devices, but at present, Xen and other virtualization methods adopt NAS,SAN and other storage modes, which are difficult to meet the needs of practical applications: on the one hand, high-speed storage devices are expensive, The energy consumption of the equipment is high and the utilization of storage space is low. On the other hand, the low fault tolerance rate of a single device is easy to cause data loss or error. In order to solve the storage problem in cloud computing virtual machine environment, many cloud storage schemes are proposed, which can provide data storage and access services by integrating many heterogeneous and cheap servers and storage devices together to provide data storage and access services. The Hadoop distributed file system (HDFS), which implements the storage management of massive data, has the advantages of high availability, high reliability, high stability and low cost, and has been widely used in many enterprises and scientific research institutions. However, the existing implementation of HDFS also has some problems such as large delay of data access, not supporting random writing and so on, which limits the application of storage support for virtual machines. Aiming at the storage problem of Xen virtual machine in cloud computing environment, this paper designs and implements a virtual platform solution based on HDFS, which extends the virtual machine mirror storage to the cloud storage platform HDFS.. The main work of this paper is as follows: (1) the performance bottleneck of virtual machine dynamic migration is analyzed deeply, and a virtual machine mirror storage framework based on HDFS architecture is proposed. A distributed fault-tolerant cloud storage mechanism which supports the dynamic running of virtual machine mirror is established. (2) based on HDFS system, a block-level storage system HBISS, which supports virtual machine mirror storage is designed by introducing the idea of log structure file system (LFS), which solves the defect that IDFS can't support random writing. Provides support for dynamic access to mirrored data. (3) according to the characteristic of Xen virtual machine accessing image file, the access mechanism from virtual machine I / o to distributed cloud storage file is established by extending the blktap tool in Xen, and the access mechanism of virtual machine to HBISS image is realized.
【學(xué)位授予單位】：南京大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2012
【分類號】：TP302;TP333
【參考文獻(xiàn)】
相關(guān)期刊論文 前2條
1 胡冷非;李小勇;;基于Xen的I/O準(zhǔn)虛擬化驅(qū)動研究[J];計算機(jī)工程;2009年23期
2 陳康;鄭緯民;;云計算:系統(tǒng)實例與研究現(xiàn)狀[J];軟件學(xué)報;2009年05期
，本文編號：2432658
本文鏈接：http://sikaile.net/kejilunwen/jisuanjikexuelunwen/2432658.html
最近更新
教材專著
