本文選題：并行計算 + 編程模型　； 參考：《上海大學》2014年博士論文

【摘要】：隨著數(shù)據(jù)規(guī)模越來越大，，復雜程度越來越高，建立大規(guī)模數(shù)據(jù)中心來對海量數(shù)據(jù)進行存儲和分析成為一個新的趨勢。為了有效地管理系統(tǒng)資源和提高系統(tǒng)可靠性，數(shù)據(jù)中心大多采用了虛擬化技術。 基于虛擬化的數(shù)據(jù)中心將大量計算、存儲和通信等資源整合到一起，以虛擬化的方式提供按需配置資源的服務模式。為了更好地利用資源，需要研究適合虛擬計算環(huán)境的并行編程模型，且編程模型應該具有性能可預測性，可以指導開發(fā)人員設計應用和配置資源。 BSP模型具有性能可預測、容易編程、能避免死鎖等優(yōu)點，BSP模型不僅適合科學計算，近幾年在并行數(shù)據(jù)庫、搜索引擎、大規(guī)模圖處理等領域獲得了廣泛應用，但是BSP模型必須與硬件結構相結合，才能充分利用硬件結構，以達到性能最優(yōu)。到目前為止，在將BSP模型與虛擬計算環(huán)境相結合，研究虛擬計算環(huán)境下基于BSP模型的大數(shù)據(jù)處理框架方面的研究還比較欠缺，本文正是基于這個目的，圍繞虛擬計算環(huán)境下性能可預測并行編程模型及其支撐技術展開，本文主要的研究成果如下： (1)針對目前面向大數(shù)據(jù)處理的并行編程模型研究中存在的不足，利用BSP模型性能可預測、易于編程、消息傳遞不產生死鎖等優(yōu)點，將BSP模型與虛擬計算環(huán)境相結合，提出一種虛擬計算環(huán)境下分布式內存與共享內存混合的并行編程模型BSPCloud。 (2)網(wǎng)絡I/O是影響B(tài)SPCloud應用性能的一個很重要的因素，在虛擬計算環(huán)境下，由于多個虛擬機共享和競爭同一I/O資源，導致虛擬機的I/O性能很難保障。本文提出一種基于I/O請求排隊的網(wǎng)絡資源調度方法。該方法根據(jù)各虛擬機的網(wǎng)絡帶寬周期性地為其分配相應的額度值，利用該額度值控制I/O請求量。通過實驗對該方法進行了性能分析和驗證，結果表明該方法能有效保障虛擬機的網(wǎng)絡I/O性能。 (3)針對虛擬計算環(huán)境下，由于VCPU調度次序的不確定性導致BSPCloud應用性能降低和性能預測能力下降的問題，提出一種基于虛擬域的VCPU協(xié)同調度方法，該方法將組調度和虛擬域相結合，可以避免VCPU調度的不確定性，提高BSPCloud應用的性能和預測能力。通過實驗對該方法進行了性能分析和驗證，結果表明該方法能有效提高BSPCloud應用性能和預測能力。 (4)在虛擬計算環(huán)境下，運行不同類型應用(比如計算密集型，I/O密集型)的虛擬機在同一物理平臺上運行時，靜態(tài)的資源分配策略不能充分利用底層物理資源。另外，BSPCloud應用將計算和通信相分離，靜態(tài)的資源分配策略會導致BSPCloud應用在計算階段浪費大量網(wǎng)絡I/O資源，而在通信階段卻浪費大量的計算資源。為了解決這個問題，本文提出一種資源動態(tài)分配方法CRDA，該方法利用虛擬機的資源分配和消耗情況來預測其資源需求，根據(jù)各虛擬機的實際需求動態(tài)分配資源。實驗結果表明該方法可以在混合負載環(huán)境下提高資源利用率，從而提高BSPCloud應用的性能。 (5)目前，基于虛擬化的資源整合變得非常流行，BSPCloud應用通常會提交到虛擬計算中心運行，為了可以預測BSPCloud在虛擬計算中心運行時的響應時間，本文對虛擬計算中心進行性能建模，將一個服務請求劃分為多個子任務，將每個子任務的處理時間劃分為計算和通信兩個階段，并充分考慮虛擬計算中心的資源調度策略和虛擬機之間的資源共享。
[Abstract]:As the scale of data is increasing and the complexity is getting higher and higher, it is a new trend to store and analyze mass data in large scale data center. In order to effectively manage system resources and improve system reliability, most data centers have adopted virtualization technology.
Data centers based on Virtualization will integrate a large number of computing, storage and communication resources together to provide a service mode for configuring resources on demand in a virtualized way. In order to make better use of resources, a parallel programming model for virtual computing environment needs to be studied. The programming model should have performance predictability and can guide development. Personnel are designed to apply and deploy resources.
The BSP model has the advantages of predictable performance, easy programming and avoiding deadlock. The BSP model is not only suitable for scientific computing, but has been widely used in the fields of parallel database, search engine and large scale graph processing in recent years, but the BSP model must be combined with the hardware structure to make full use of the hardware structure to achieve the best performance. So far, combining the BSP model with the virtual computing environment, the research on the large data processing framework based on the BSP model in the virtual computing environment is still relatively short. This paper is based on this purpose. This paper focuses on the performance predictable parallel programming model and its supporting technology under the virtual computing environment. The main research results of this paper are as follows. Below:
(1) in view of the shortcomings of the parallel programming model for large data processing, the advantages of the BSP model can be predicted, easy to program, and the message transfer does not produce the life and death lock. The BSP model is combined with the virtual computing environment, and a parallel programming model, BSP, which is mixed with distributed memory and shared memory under the virtual computing environment, is proposed. Cloud.
(2) network I/O is an important factor affecting the performance of BSPCloud applications. In the virtual computing environment, because of the sharing and competition of the same I/O resources by multiple virtual machines, the I/O performance of the virtual machine is difficult to guarantee. This paper proposes a network resource adjustment method based on I/O request queuing. This method is based on the network bandwidth cycle of each virtual machine. The corresponding quota is allocated for it in a period of time, and the amount of the I/O request is controlled by the amount. The performance analysis and verification of the method is carried out through experiments. The results show that the method can effectively protect the network I/O performance of the virtual machine.
(3) in the virtual computing environment, due to the uncertainty of VCPU scheduling order resulting in the degradation of BSPCloud application performance and the decline of performance prediction ability, a collaborative scheduling method based on virtual domain based VCPU is proposed. This method combines group scheduling and virtual domain to avoid the uncertainty of VCPU scheduling and improve the application of BSPCloud applications. The performance of the method is analyzed and verified through experiments. The results show that the method can effectively improve the performance and prediction ability of BSPCloud applications.
(4) under the virtual computing environment, when running different types of applications (such as computing intensive, I/O intensive) virtual machines running on the same physical platform, the static resource allocation strategy can not make full use of the underlying physical resources. In addition, the BSPCloud application separates the computing and communication, and the static resource allocation strategy will lead to the BSPCloud application A large amount of network I/O resources are wasted in the computing stage, but a lot of computing resources are wasted in the communication stage. In order to solve this problem, a dynamic resource allocation method, CRDA, is proposed in this paper. This method uses the resource allocation and consumption of virtual machines to predict the resource requirements, and the resource allocation is dynamically allocated according to the actual requirements of each virtual machine. The test results show that this method can improve resource utilization in mixed load environment, thereby improving the performance of BSPCloud applications.
(5) at present, resource integration based on virtualization has become very popular. BSPCloud applications are usually submitted to virtual computing centers. In order to predict the response time of BSPCloud in the virtual computing center, the performance modeling of the virtual computing center is made, and a service request is divided into multiple subtasks and each sub task is made. The processing time is divided into two stages of computation and communication, and the resource scheduling strategy of virtual computing center and the resource sharing between virtual machines are fully considered.

【學位授予單位】：上海大學
【學位級別】：博士
【學位授予年份】：2014
【分類號】：TP391.9

【參考文獻】

相關期刊論文 前10條

1 金海;鐘阿林;吳松;石宣化;;多核環(huán)境下虛擬機VCPU調度研究:問題與挑戰(zhàn)[J];計算機研究與發(fā)展;2011年07期

2 王凱;侯紫峰;;Xen虛擬機的虛擬CPU松弛協(xié)同調度方法[J];計算機研究與發(fā)展;2012年01期

3 陸維明;;Petri網(wǎng)與DNA計算[J];計算機科學;1998年01期

4 王珊;王會舉;覃雄派;周p

本文編號：1780933