本文關(guān)鍵詞：基于粒子群調(diào)度器的云存儲(chǔ)系統(tǒng)針對(duì)交互密集型任務(wù)的緩存模型研究　出處：《華東師范大學(xué)》2012年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 分布式文件系統(tǒng) HDFS 分布式緩存 粒子群 云計(jì)算

【摘要】：隨著計(jì)算機(jī)和網(wǎng)絡(luò)技術(shù)日益先進(jìn),以及當(dāng)今社會(huì)對(duì)對(duì)計(jì)算資源的需求日益旺盛,許多公司廠商和研究機(jī)構(gòu)開(kāi)始探求通過(guò)互聯(lián)網(wǎng)絡(luò)向外部用戶租售輸出存儲(chǔ)和計(jì)算能力的可能性。這種以經(jīng)濟(jì)需求為導(dǎo)向的外向型計(jì)算模式在今天被稱作云計(jì)算模式。而云計(jì)算模式實(shí)現(xiàn)的基礎(chǔ),則是大型的計(jì)算和存儲(chǔ)集群。 在另一個(gè)方面,云計(jì)算的創(chuàng)新性也給其相關(guān)技術(shù)提出了嚴(yán)峻的挑戰(zhàn)和嶄新的問(wèn)題。比如,其巨大的規(guī)模已經(jīng)使計(jì)算集群在很多方面產(chǎn)生了亟待突破的性能瓶頸,而本文所研究的云存儲(chǔ)系統(tǒng)在處理交互密集型任務(wù)時(shí)會(huì)產(chǎn)生巨大的性能損益這一問(wèn)題便是其中之一。 本文的研究依托于Hadoop Distribute File System分布式文件系統(tǒng)所構(gòu)建的分布式存儲(chǔ)平臺(tái),提出了一種基于粒子群調(diào)度分配算法的主從名字節(jié)點(diǎn)緩沖架構(gòu),旨在解決原有系統(tǒng)在面對(duì)具有頻繁寫入讀出特性的一類用戶應(yīng)用程序時(shí)系統(tǒng)吞吐量劇烈降低這一問(wèn)題。該架構(gòu)在結(jié)構(gòu)上對(duì)原分布式文件系統(tǒng)做出了兩項(xiàng)修改。首先,將原分布式文件系統(tǒng)的單名字節(jié)點(diǎn)結(jié)構(gòu)改變?yōu)殡p層主從名字節(jié)點(diǎn)結(jié)構(gòu)。然后再在每個(gè)從名字節(jié)點(diǎn)上實(shí)現(xiàn)基于tmpfs文件系統(tǒng)的數(shù)據(jù)塊緩存機(jī)制。同時(shí),該架構(gòu)還采用了基于粒子群算法的文件塊調(diào)度措施,用來(lái)將到來(lái)的文件塊優(yōu)化分配到合適的數(shù)據(jù)節(jié)點(diǎn)或緩存中。由于單純的粒子群算法在本文所提系統(tǒng)的環(huán)境中極易陷入局部最優(yōu)解而對(duì)整個(gè)系統(tǒng)的性能造成較大的負(fù)面印象,所以本文對(duì)粒子群算法進(jìn)行了修改,使之具有進(jìn)化程度評(píng)估和參數(shù)自適應(yīng)調(diào)整的能力。 本文所提出的架構(gòu)在一個(gè)130個(gè)節(jié)點(diǎn)的松耦合分布式環(huán)境中進(jìn)行了實(shí)際地部署和驗(yàn)證,并針對(duì)交互密集型應(yīng)用的特點(diǎn)進(jìn)行了一系列測(cè)試。實(shí)驗(yàn)結(jié)果證明,本文所采用的具有進(jìn)化程度評(píng)估和參數(shù)自適應(yīng)調(diào)整能力的粒子群優(yōu)化調(diào)度有效地提高了算法的效率并降低了調(diào)度方案陷入局部最優(yōu)解的問(wèn)題。同時(shí),針對(duì)多項(xiàng)指標(biāo)的綜合測(cè)試實(shí)驗(yàn)證明,本文提出的基于粒子群調(diào)度的兩層緩沖模型能夠切實(shí)地解決分布式文件系統(tǒng)面對(duì)交互密集型應(yīng)用所帶來(lái)的系統(tǒng)吞吐量下降的問(wèn)題。
[Abstract]:With the increasing sophistication of computer and networking technologies , and the growing demand for computing resources in today ' s society , many companies and research institutes have begun to explore the possibility of renting out output storage and computing power through the Internet to external users . Such export - oriented computing models guided by economic demand are today called cloud computing models . The foundation for cloud computing models is large computing and storage clusters . In another aspect , the innovative nature of cloud computing poses a serious challenge and a new problem to its related technologies . For example , its massive scale has made computing clusters a critical performance bottleneck in many ways , and the problem of cloud storage systems , as discussed herein , results in significant performance gains and losses when dealing with interactive intensive tasks . Based on the distributed storage platform built by the distributed file system of Hadoop distributed file system , this paper proposes a master - slave name node buffer architecture based on the particle swarm scheduling algorithm . It is designed to solve the problem of drastic reduction in throughput of the original distributed file system . At the same time , the structure of the single - name node of the original distributed file system is changed to double - layer master - slave name node structure . In this paper , a series of tests are carried out in a loosely coupled distributed environment of 130 nodes , and a series of tests are carried out for the characteristics of interaction - intensive applications . The experimental results show that the particle swarm optimization scheduling with the evolutionary degree evaluation and the adaptive adjustment capability of the parameters effectively improves the efficiency of the algorithm and reduces the problem that the scheduling scheme is trapped in the local optimal solution .

【學(xué)位授予單位】：華東師范大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2012
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前4條

1 陳玉蘭;鄭駿;胡文心;;一種多QoS約束的網(wǎng)格資源調(diào)度算法[J];華東師范大學(xué)學(xué)報(bào)(自然科學(xué)版);2009年01期

2 楊曉帆;陳廷槐;;人工神經(jīng)網(wǎng)絡(luò)固有的優(yōu)點(diǎn)和缺點(diǎn)[J];計(jì)算機(jī)科學(xué);1994年02期

3 張娜;鄭駿;;基于三角網(wǎng)格請(qǐng)求集的動(dòng)態(tài)位置管理算法[J];計(jì)算機(jī)工程;2007年22期

4 王佳莉;鄭駿;胡文心;蔡建華;杜平;;分布式異構(gòu)數(shù)據(jù)網(wǎng)絡(luò)信息平臺(tái)的應(yīng)用研究[J];計(jì)算機(jī)與數(shù)字工程;2008年08期

，

本文編號(hào)：1386527