本文選題：并行網(wǎng)絡文件系統(tǒng) + 負載均衡　； 參考：《華中科技大學》2012年碩士論文

【摘要】：負載均衡技術是并行文件系統(tǒng)中常見且不可或缺的重要優(yōu)化手段。在并行文件系統(tǒng)中實現(xiàn)副本技術，利用負載分析方法和調度算法，可以有效地將系統(tǒng)中各種負載均衡地分配到各存儲節(jié)點上，以提高文件系統(tǒng)的可用性、穩(wěn)定性和服務質量。 在并行文件系統(tǒng)中，隨著節(jié)點的增加、刪除，文件的創(chuàng)建、刪除和修改，以及訪問次數(shù)的爆炸性增長，各個存儲節(jié)點的負載以不可預知的方式動態(tài)變化，常常發(fā)生部分服務器資源損耗過度而另外一些服務器使用率低下的情況，即負載傾斜。同時，節(jié)點的突發(fā)故障極易導致系統(tǒng)無法正常運行。雖然使用副本機制有助于解決上述問題，但在不同場景下，還需要針對場景中的I/O行為特征，，以副本為基礎，圍繞負載均衡技術進行研究，找出適用的策略和時機。本文針對一種典型的并行文件系統(tǒng)pNFS（Parallel Network File System），設計并實現(xiàn)了基于概率分布的動態(tài)負載均衡機制PDDB（Probability Distribution Dynamic Balance）。 在并行網(wǎng)絡文件系統(tǒng)中，PDDB通過對文件創(chuàng)建副本，按照容量均衡的方式進行文件副本的合理放置，同時設計了鏡像和交錯的兩種副本放置模式。PDDB在各數(shù)據(jù)服務器上構建了自適應的負載監(jiān)控系統(tǒng)，由元數(shù)據(jù)服務器收集各個節(jié)點的CPU、內存、存儲空間、網(wǎng)絡帶寬、磁盤帶寬等負載信息，經(jīng)過對負載信息的整合處理，并結合已經(jīng)獲得的歷史數(shù)據(jù)信息，以綜合負載的大小決定任務分配的概率，將訪問均衡地分配到當前負載較低的一組服務器上，并通過熱點遷移調度各個節(jié)點之間的負載狀態(tài)，避免群聚效應。并加入副本元數(shù)據(jù)的管理以及副本一致性的維護，保證當任一副本的存儲節(jié)點發(fā)生故障時，可以通過其他副本正常獲取文件內容，確保系統(tǒng)的正常運行。 在測試中發(fā)現(xiàn)，與使用隨機算法和最小負載優(yōu)先算法的負載均衡機制相比較，PDDB機制使得各存儲節(jié)點之間的最大負載差距比二者分別減少了42%和30%，且在文件系統(tǒng)總負載相同時，各個節(jié)點的負載變化也相對平緩，平均負載至少降低10%以上。文件系統(tǒng)的平均網(wǎng)絡吞吐率提升20%，可靠性、穩(wěn)定性和可擴展性均得到改善。
[Abstract]:Load balancing is a common and indispensable optimization method in parallel file systems. By using load analysis method and scheduling algorithm, replicas in parallel file systems can be effectively distributed to each storage node in order to improve the availability, stability and quality of service of the file system. In parallel file system, with the increase of nodes, deletion, creation, deletion and modification of files, and the explosive increase of access, the load of each storage node changes in an unpredictable way. Some server resources are overused and others are underutilized, that is, load tilting. At the same time, the sudden failure of nodes can easily lead to the normal operation of the system. Although using replica mechanism is helpful to solve the above problems, it is necessary to study the load balancing technology based on replicas in different scenarios, and find out the appropriate strategies and opportunities according to the behavior characteristics of I / O in different scenarios. For a typical parallel file system, pNFS(Parallel Network File system, a dynamic load balancing mechanism based on probability distribution, PDDB(Probability Distribution Dynamic balance, is designed and implemented in this paper. In parallel network file system, PDDB can make a copy of the file by making a copy of the file, and make a reasonable arrangement of the copy of the file according to the way of capacity balance. At the same time, two replica placement modes, mirroring and interleaving, are designed. PDDB constructs an adaptive load monitoring system on each data server. The metadata server collects the CPU, memory, storage space and network bandwidth of each node. The load information, such as disk bandwidth, is processed by the integration of the load information, and combined with the historical data obtained, the probability of task allocation is determined by the size of the comprehensive load. Access is distributed evenly to a group of servers with low current load, and the load state between nodes is scheduled by hot spot migration to avoid clustering effect. The management of replica metadata and the maintenance of replica consistency are added to ensure that the contents of files can be obtained through other replicas when any storage node of the replica fails to ensure the normal operation of the system. It was found in the test that the PDDB mechanism reduced the maximum load gap between storage nodes by 42% and 30% respectively compared with the load balancing mechanism using random algorithm and minimum load first algorithm, and when the total load of the file system was the same, the PDDB mechanism reduced the maximum load gap between each storage node by 42% and 30% respectively, and when the total load of the file system was the same, The load change of each node is relatively smooth, the average load is reduced by more than 10%. The average network throughput of the file system is increased by 20%, and the reliability, stability and scalability are improved.
【學位授予單位】：華中科技大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP338.6

【參考文獻】

相關期刊論文 前4條

1 龐麗萍,許俊,徐婕,岳建輝;PVFS數(shù)據(jù)訪問的負載平衡[J];華中科技大學學報(自然科學版);2004年07期

2 龔梅;王鵬;吳躍;;一種集群系統(tǒng)的透明動態(tài)反饋負載均衡算法[J];計算機應用;2007年11期

3 張媛;于冠龍;盧澤新;劉亞萍;;并行網(wǎng)絡文件系統(tǒng)PNFS性能評測與分析[J];計算機工程與應用;2009年35期

4 陳志剛,李登,曾志文;分布式系統(tǒng)中一種動態(tài)負載均衡策略、相關模型及算法研究[J];小型微型計算機系統(tǒng);2002年12期

本文編號：1862609