本文選題：云計算 + 大數(shù)據(jù)��； 參考：《電子科技大學(xué)》2017年碩士論文

【摘要】：隨著大量的基于互聯(lián)網(wǎng)的服務(wù)與大量的服務(wù)托管在云平臺上的趨勢日益流行,需要更加強大的后端存儲系統(tǒng)來支持這些服務(wù)。一方面存儲系統(tǒng)自身應(yīng)該要有更強大的處理高并發(fā)和高強度的工作負載的能力。分布式存儲是當(dāng)前一種很優(yōu)秀的解決辦法。它將會實現(xiàn)以下特性:伸縮性、可用性、持久性、一致性以及分區(qū)容忍性。目前實現(xiàn)一個分布式存儲來滿足以上所有設(shè)想是非常困難的或者說是不可能的。因此,把研究的重點放在限制不同的特性來設(shè)計不同的分布式存儲解決方案來滿足不同的使用場景。另一方面,隨著云計算技術(shù)的發(fā)展為設(shè)計分布式存儲帶來了新的挑戰(zhàn)。為了更好的在云計算中使用現(xiàn)收現(xiàn)付的價格模型,一種動態(tài)的資源部署中間件被提出來(如彈性控制器)。彈性控制是能夠幫助節(jié)約應(yīng)用的程序的配置成本,同時不損耗應(yīng)用的性能。設(shè)計一種彈性控制機制來自動部署分布式存儲系統(tǒng)是非常重要的。這個問題將要面對的挑戰(zhàn)是對系統(tǒng)的工作負載的正確判斷和在伸縮系統(tǒng)的時候的數(shù)據(jù)遷移開銷(即是系統(tǒng)添加或刪除實例的時候傳輸數(shù)據(jù)消耗的時間和資源)。本課題設(shè)計的自動部署系統(tǒng)主要是采用了工作負載預(yù)測的方法來實現(xiàn)的。主要是通過監(jiān)測存儲系統(tǒng)的性能指標數(shù)據(jù)(請求延遲、cpu利用率、實例數(shù)量等),通過檢測的工作負載數(shù)據(jù)來預(yù)測系統(tǒng)下一個時期的工作負載情況,以及使用檢測的數(shù)據(jù)推出系統(tǒng)滿足SLA的最小配置,從而在下一預(yù)測窗口開始前重新部署系統(tǒng)規(guī)模。在添加或移除實例的時候會做一個數(shù)據(jù)的遷移,使服務(wù)請求更加均勻的訪問每一個存儲實例。對工作負載的預(yù)測是整個系統(tǒng)的核心部分,本課題采用了基于維納濾波器的原理實現(xiàn)的維納預(yù)測器來預(yù)測每一個工作負載預(yù)測窗口的工作負載,這也是本課題的創(chuàng)新點。對于預(yù)測工作負載本文采用特定類型的工作負載,根據(jù)其周期性將工作負載分為特定時長的預(yù)測窗口,在每一個預(yù)測窗口開始的時候預(yù)測下一個預(yù)測窗口的工作負載。最后,通過大量的實驗和測試對本課題提出的提出的預(yù)測算法以及驗證分布式系統(tǒng)的性能,證實了基于維納預(yù)測的方案能夠在保證SLA的前提下,節(jié)約平臺資源同時也就節(jié)省了配置成本。
[Abstract]:With the increasing popularity of a large number of Internet-based services and a large number of services hosted on cloud platforms, more powerful back-end storage systems are needed to support these services. On the one hand, storage systems themselves should be more powerful in handling high concurrent and high-intensity workloads. Distributed storage is currently an excellent solution. It will implement the following features: scalability, availability, persistence, consistency, and partition tolerance. It is very difficult or impossible to implement a distributed storage to meet all of the above assumptions. Therefore, the research focuses on limiting different features to design different distributed storage solutions to meet different usage scenarios. On the other hand, with the development of cloud computing technology, it brings new challenges to the design of distributed storage. In order to better use pay-as-you-go model in cloud computing, a dynamic resource deployment middleware (such as flexible controller) is proposed. Flexible control is a program that can help save application configuration costs without loss of application performance. It is very important to design an elastic control mechanism to deploy distributed storage system automatically. The challenge of this problem is to correctly judge the workload of the system and the data migration overhead (i.e., the time and resources of transferring data when the system adds or deletes an instance). The automatic deployment system designed in this paper mainly adopts the method of workload prediction. Mainly by monitoring the performance index data of the storage system (request delay CPU utilization, the number of instances, etc.), the workload data is detected to predict the workload of the system in the next period. Using the detected data, the system meets the minimum configuration of SLA to redeploy the system scale before the next prediction window begins. When an instance is added or removed, a data migration is made, which makes the service request more evenly access each storage instance. The prediction of workload is the core of the whole system. In this paper, a Wiener predictor based on the principle of Wiener filter is used to predict the workload of every workload prediction window, which is also the innovation of this subject. In this paper, a specific type of workload is used to predict the workload. According to its periodicity, the workload is divided into prediction windows of specific duration, and the workload of the next prediction window is predicted at the beginning of each prediction window. Finally, through a large number of experiments and tests on the proposed prediction algorithm and verify the performance of the distributed system, it is proved that the scheme based on Wiener prediction can guarantee SLA. Save platform resource at the same time also save configuration cost.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP333;TP311.13

【參考文獻】

相關(guān)期刊論文 前1條

1 阮夢黎;;大數(shù)據(jù)挑戰(zhàn)下的NoSQL系統(tǒng)研究[J];聊城大學(xué)學(xué)報(自然科學(xué)版);2015年01期

相關(guān)碩士學(xué)位論文 前1條

1 劉龍;基于云計算的nosql集群自動資源配置系統(tǒng)的研究與實現(xiàn)[D];西安工程大學(xué);2015年

，

本文編號：2067689