本文選題：云計(jì)算 + 大數(shù)據(jù)��； 參考：《電子科技大學(xué)》2017年碩士論文

【摘要】：隨著大量的基于互聯(lián)網(wǎng)的服務(wù)與大量的服務(wù)托管在云平臺(tái)上的趨勢(shì)日益流行,需要更加強(qiáng)大的后端存儲(chǔ)系統(tǒng)來(lái)支持這些服務(wù)。一方面存儲(chǔ)系統(tǒng)自身應(yīng)該要有更強(qiáng)大的處理高并發(fā)和高強(qiáng)度的工作負(fù)載的能力。分布式存儲(chǔ)是當(dāng)前一種很優(yōu)秀的解決辦法。它將會(huì)實(shí)現(xiàn)以下特性:伸縮性、可用性、持久性、一致性以及分區(qū)容忍性。目前實(shí)現(xiàn)一個(gè)分布式存儲(chǔ)來(lái)滿足以上所有設(shè)想是非常困難的或者說(shuō)是不可能的。因此,把研究的重點(diǎn)放在限制不同的特性來(lái)設(shè)計(jì)不同的分布式存儲(chǔ)解決方案來(lái)滿足不同的使用場(chǎng)景。另一方面,隨著云計(jì)算技術(shù)的發(fā)展為設(shè)計(jì)分布式存儲(chǔ)帶來(lái)了新的挑戰(zhàn)。為了更好的在云計(jì)算中使用現(xiàn)收現(xiàn)付的價(jià)格模型,一種動(dòng)態(tài)的資源部署中間件被提出來(lái)(如彈性控制器)。彈性控制是能夠幫助節(jié)約應(yīng)用的程序的配置成本,同時(shí)不損耗應(yīng)用的性能。設(shè)計(jì)一種彈性控制機(jī)制來(lái)自動(dòng)部署分布式存儲(chǔ)系統(tǒng)是非常重要的。這個(gè)問(wèn)題將要面對(duì)的挑戰(zhàn)是對(duì)系統(tǒng)的工作負(fù)載的正確判斷和在伸縮系統(tǒng)的時(shí)候的數(shù)據(jù)遷移開(kāi)銷(xiāo)(即是系統(tǒng)添加或刪除實(shí)例的時(shí)候傳輸數(shù)據(jù)消耗的時(shí)間和資源)。本課題設(shè)計(jì)的自動(dòng)部署系統(tǒng)主要是采用了工作負(fù)載預(yù)測(cè)的方法來(lái)實(shí)現(xiàn)的。主要是通過(guò)監(jiān)測(cè)存儲(chǔ)系統(tǒng)的性能指標(biāo)數(shù)據(jù)(請(qǐng)求延遲、cpu利用率、實(shí)例數(shù)量等),通過(guò)檢測(cè)的工作負(fù)載數(shù)據(jù)來(lái)預(yù)測(cè)系統(tǒng)下一個(gè)時(shí)期的工作負(fù)載情況,以及使用檢測(cè)的數(shù)據(jù)推出系統(tǒng)滿足SLA的最小配置,從而在下一預(yù)測(cè)窗口開(kāi)始前重新部署系統(tǒng)規(guī)模。在添加或移除實(shí)例的時(shí)候會(huì)做一個(gè)數(shù)據(jù)的遷移,使服務(wù)請(qǐng)求更加均勻的訪問(wèn)每一個(gè)存儲(chǔ)實(shí)例。對(duì)工作負(fù)載的預(yù)測(cè)是整個(gè)系統(tǒng)的核心部分,本課題采用了基于維納濾波器的原理實(shí)現(xiàn)的維納預(yù)測(cè)器來(lái)預(yù)測(cè)每一個(gè)工作負(fù)載預(yù)測(cè)窗口的工作負(fù)載,這也是本課題的創(chuàng)新點(diǎn)。對(duì)于預(yù)測(cè)工作負(fù)載本文采用特定類(lèi)型的工作負(fù)載,根據(jù)其周期性將工作負(fù)載分為特定時(shí)長(zhǎng)的預(yù)測(cè)窗口,在每一個(gè)預(yù)測(cè)窗口開(kāi)始的時(shí)候預(yù)測(cè)下一個(gè)預(yù)測(cè)窗口的工作負(fù)載。最后,通過(guò)大量的實(shí)驗(yàn)和測(cè)試對(duì)本課題提出的提出的預(yù)測(cè)算法以及驗(yàn)證分布式系統(tǒng)的性能,證實(shí)了基于維納預(yù)測(cè)的方案能夠在保證SLA的前提下,節(jié)約平臺(tái)資源同時(shí)也就節(jié)省了配置成本。
[Abstract]:With the increasing popularity of a large number of Internet-based services and a large number of services hosted on cloud platforms, more powerful back-end storage systems are needed to support these services. On the one hand, storage systems themselves should be more powerful in handling high concurrent and high-intensity workloads. Distributed storage is currently an excellent solution. It will implement the following features: scalability, availability, persistence, consistency, and partition tolerance. It is very difficult or impossible to implement a distributed storage to meet all of the above assumptions. Therefore, the research focuses on limiting different features to design different distributed storage solutions to meet different usage scenarios. On the other hand, with the development of cloud computing technology, it brings new challenges to the design of distributed storage. In order to better use pay-as-you-go model in cloud computing, a dynamic resource deployment middleware (such as flexible controller) is proposed. Flexible control is a program that can help save application configuration costs without loss of application performance. It is very important to design an elastic control mechanism to deploy distributed storage system automatically. The challenge of this problem is to correctly judge the workload of the system and the data migration overhead (i.e., the time and resources of transferring data when the system adds or deletes an instance). The automatic deployment system designed in this paper mainly adopts the method of workload prediction. Mainly by monitoring the performance index data of the storage system (request delay CPU utilization, the number of instances, etc.), the workload data is detected to predict the workload of the system in the next period. Using the detected data, the system meets the minimum configuration of SLA to redeploy the system scale before the next prediction window begins. When an instance is added or removed, a data migration is made, which makes the service request more evenly access each storage instance. The prediction of workload is the core of the whole system. In this paper, a Wiener predictor based on the principle of Wiener filter is used to predict the workload of every workload prediction window, which is also the innovation of this subject. In this paper, a specific type of workload is used to predict the workload. According to its periodicity, the workload is divided into prediction windows of specific duration, and the workload of the next prediction window is predicted at the beginning of each prediction window. Finally, through a large number of experiments and tests on the proposed prediction algorithm and verify the performance of the distributed system, it is proved that the scheme based on Wiener prediction can guarantee SLA. Save platform resource at the same time also save configuration cost.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類(lèi)號(hào)】：TP333;TP311.13

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 阮夢(mèng)黎;;大數(shù)據(jù)挑戰(zhàn)下的NoSQL系統(tǒng)研究[J];聊城大學(xué)學(xué)報(bào)(自然科學(xué)版);2015年01期

相關(guān)碩士學(xué)位論文 前1條

1 劉龍;基于云計(jì)算的nosql集群自動(dòng)資源配置系統(tǒng)的研究與實(shí)現(xiàn)[D];西安工程大學(xué);2015年

，

本文編號(hào)：2067689