本文選題：云計(jì)算 + MapReduce�。� 參考：《吉林大學(xué)》2013年碩士論文

【摘要】：云計(jì)算自2006年提出以來，改變了人們對(duì)于網(wǎng)絡(luò)的概念，目前廣泛應(yīng)用于互聯(lián)網(wǎng)各個(gè)方面。目前各個(gè)IT行業(yè)領(lǐng)導(dǎo)者們都在部署和研究云計(jì)算技術(shù)，把云計(jì)算技術(shù)應(yīng)用于核心服務(wù)中。云計(jì)算是新一代互聯(lián)網(wǎng)技術(shù)的研究熱點(diǎn)。因此對(duì)于云計(jì)算技術(shù)的研究有著重要的實(shí)際意義。 云計(jì)算技術(shù)的關(guān)鍵之一是由Google提出來的MapReduce并行數(shù)據(jù)編程模型框架。Hadoop平臺(tái)是對(duì)于Google公司的MapReduce的一種開源模仿，也是世界上應(yīng)用最為廣泛的一種開源云計(jì)算平臺(tái)。因此對(duì)于Hadoop平臺(tái)下的MapReduce模型的研究和優(yōu)化有著重要的意義。 本文首先介紹云計(jì)算的概念和背景知識(shí)，然后介紹并分析Hadoop平臺(tái)的關(guān)鍵技術(shù)。研究Hadoop平臺(tái)中的MapReduce模型的機(jī)制之后，針對(duì)存在的不足之處，提出了一種改進(jìn)的方案，命名為動(dòng)態(tài)自適應(yīng)調(diào)度算法（Adaptive CapacityAlgorithm Based on Priority，以下簡稱ACBP）。本文算法在運(yùn)行中按照實(shí)際運(yùn)行情況動(dòng)態(tài)改變?cè)O(shè)置的執(zhí)行作業(yè)數(shù)量，實(shí)現(xiàn)自適應(yīng)的系統(tǒng)任務(wù)調(diào)度機(jī)制，而且本文對(duì)Hadoop默認(rèn)的推測(cè)機(jī)制進(jìn)行了研究，針對(duì)推測(cè)機(jī)制的不足之處進(jìn)行改進(jìn)，使得判別落后的任務(wù)更為準(zhǔn)側(cè)，避免不必要的系統(tǒng)計(jì)算資源的消耗。啟動(dòng)備份任務(wù)的時(shí)候，對(duì)節(jié)點(diǎn)的分配上，，有必要考慮節(jié)點(diǎn)的系統(tǒng)負(fù)載情況，考慮剩余計(jì)算能力，來合理分配備份任務(wù)運(yùn)行節(jié)點(diǎn)，避免啟動(dòng)無效備份任務(wù)，從而有效提高系統(tǒng)整 本文最后通過實(shí)驗(yàn)驗(yàn)證本算法在預(yù)期條件下的算法性能，和先進(jìn)先出調(diào)度算法和公平調(diào)度算法和能力調(diào)度算法經(jīng)過對(duì)比，得出實(shí)驗(yàn)結(jié)果。實(shí)驗(yàn)結(jié)果表明本文的算法比先進(jìn)先出調(diào)度算法有著更為良好的性能，但是相比公平調(diào)度算法和能力調(diào)度算法，存在著一定的局限性。但是在特定環(huán)境中有著預(yù)期的的表現(xiàn)。實(shí)現(xiàn)了實(shí)驗(yàn)的預(yù)期目的。
[Abstract]:Cloud computing has changed the concept of network since it was put forward in 2006, and has been widely used in all aspects of the Internet. At present, various IT industry leaders are deploying and researching cloud computing technology and applying cloud computing technology to core services. Cloud computing is the research hotspot of the new generation Internet technology. Therefore, the research of cloud computing technology has important practical significance. One of the key technologies of cloud computing is that the MapReduce parallel data programming model framework. Hadoop platform, which is proposed by Google, is an open source imitation of MapReduce made by Google, and it is also the most widely used open source cloud computing platform in the world. Therefore, it is of great significance for the research and optimization of MapReduce model based on Hadoop platform. This paper first introduces the concept and background of cloud computing, then introduces and analyzes the key technologies of Hadoop platform. After studying the mechanism of MapReduce model in Hadoop platform, an improved scheme named Adaptive capacity algorithm based on Priorityis proposed. In this paper, according to the actual running situation, the algorithm dynamically changes the number of execution jobs and realizes the adaptive system task scheduling mechanism. Furthermore, the default mechanism of Hadoop is studied in this paper. Aiming at the inadequacies of the speculate mechanism, it can make the judgment of backward tasks more accurate and avoid the unnecessary consumption of system computing resources. When the backup task is started, it is necessary to consider the system load of the node and the residual computing ability to distribute the backup task running node reasonably, to avoid starting the invalid backup task. Finally, the performance of the proposed algorithm is verified by experiments, and compared with the first-in-first-out scheduling algorithm, fair scheduling algorithm and ability scheduling algorithm, the experimental results are obtained. The experimental results show that the proposed algorithm has better performance than the first-in-first-out scheduling algorithm, but it has some limitations compared with the fair scheduling algorithm and the ability scheduling algorithm. But in a particular environment there is expected performance. The expected purpose of the experiment has been achieved.
【學(xué)位授予單位】：吉林大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP338

【參考文獻(xiàn)】

相關(guān)期刊論文 前3條

1 陳全;鄧倩妮;;云計(jì)算及其關(guān)鍵技術(shù)[J];計(jì)算機(jī)應(yīng)用;2009年09期

2 周鋒;李旭偉;;一種改進(jìn)的MapReduce并行編程模型[J];科協(xié)論壇(下半月);2009年02期

3 孫廣中;肖鋒;熊曦;;MapReduce模型的調(diào)度及容錯(cuò)機(jī)制研究[J];微電子學(xué)與計(jì)算機(jī);2007年09期

本文編號(hào)：2051414