本文選題：分布式 + 計算框架��； 參考：《華中科技大學》2012年碩士論文

【摘要】：近年來隨著計算機網(wǎng)絡技術(shù)的高速發(fā)展，互聯(lián)網(wǎng)企業(yè)為用戶提供的應用和個性化服務也越來越多，隨之而來的是用戶提交的海量數(shù)據(jù)，企業(yè)必須及時而又正確地處理這些數(shù)據(jù)并向用戶反饋結(jié)果。通常，人們會選擇使用分布式計算系統(tǒng)來完成這些復雜的任務。然而傳統(tǒng)的分布式計算平臺往往需要配置大型的服務器集群，還需要專門的技術(shù)人員去跟進和維護，這給很多企業(yè)都帶來了巨大的經(jīng)濟開銷。 輕量級分布式任務計算框架Antnest采用了客戶機/服務器計算模式和三層體系結(jié)構(gòu)，可以將工作任務分布在廉價的任務節(jié)點上，中心節(jié)點負責整個系統(tǒng)的調(diào)度工作，任務節(jié)點采用多JVM多線程執(zhí)行任務。在Antnest中，提交的任務都是經(jīng)過定制的，不同的任務對實時性的要求各不相同，針對此，Antnest為任務調(diào)度系統(tǒng)設計了一種多級隊列搶占式調(diào)度算法，該算法充分考慮到任務對實時性的要求，高優(yōu)先級任務可以搶占系統(tǒng)，被搶占任務會移入高優(yōu)先級隊列中等待調(diào)度，同時利用任務的預定啟動時間來確保任務執(zhí)行的有效性。另外，Antnest還設計了動態(tài)任務分配算法來完成子任務的分配工作。在動態(tài)分配算法中，子任務和任務節(jié)點的分配關系是在任務的執(zhí)行過程中動態(tài)確定的，Antnest會優(yōu)先將子任務分配給最先執(zhí)行完子任務的任務節(jié)點。在任務調(diào)度系統(tǒng)的實現(xiàn)中，任務調(diào)度系統(tǒng)各模塊間通過自定義的通信協(xié)議完成消息通信，，通過指定的任務接口觸發(fā)任務邏輯，利用任務節(jié)點的狀態(tài)管理機制完成資源管理并保證Antnest的可擴展性，同時，利用心跳檢測和子任務重傳的容錯機制確保任務的正確執(zhí)行。 測試結(jié)果表明，Antnest系統(tǒng)能有效地提高任務的執(zhí)行效率，減短執(zhí)行時間；同時，結(jié)果也驗證了Antnest系統(tǒng)中多級隊列搶占式調(diào)度算法的合理性和可行性。
[Abstract]:In recent years, with the rapid development of computer network technology, Internet enterprises provide users with more and more applications and personalized services, followed by massive data submitted by users. The enterprise must process the data in a timely and correct manner and feedback the results to the user. Usually, people choose to use distributed computing systems to accomplish these complex tasks. However, traditional distributed computing platforms often need to configure large server clusters, and also need specialized technicians to follow up and maintain, which brings huge economic expenses to many enterprises. The lightweight distributed task computing framework (Antnest) adopts the client / server computing model and three-tier architecture, which can distribute the work tasks on the cheap task nodes, and the central node is responsible for the scheduling of the whole system. The task node uses multi-JVM multi-thread to execute the task. In Antnest, all the submitted tasks are customized, and different tasks require different real-time performance. A multi-level queue preemptive scheduling algorithm is designed for this task scheduling system. The algorithm fully takes into account the real-time requirements of tasks, high priority tasks can preemptive system, preemptive tasks will be moved into high priority queue to wait for scheduling, and the scheduled start time of tasks will be used to ensure the effectiveness of task execution. In addition, Antnest designs a dynamic task allocation algorithm to accomplish the task assignment. In the dynamic assignment algorithm, the relationship between subtasks and task nodes is dynamically determined in the process of task execution. In the implementation of task scheduling system, message communication is accomplished between modules of task scheduling system through self-defined communication protocol, and task logic is triggered by specified task interface. The state management mechanism of the task node is used to accomplish resource management and to ensure the scalability of the Antnest. At the same time, the fault-tolerant mechanism of heartbeat detection and sub-task retransmission is used to ensure the correct execution of the task. The test results show that the Antnest system can effectively improve the task execution efficiency and reduce the execution time, and it also verifies the rationality and feasibility of the multilevel queue preemptive scheduling algorithm in Antnest system.
【學位授予單位】：華中科技大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP338.8

【參考文獻】

相關期刊論文 前10條

1 李本亮;王厚軍;李力;;基于WLAN的分布式測試系統(tǒng)實時性研究[J];電子測量與儀器學報;2008年06期

2 趙明宇;張?zhí)镂?;一種分布式計算環(huán)境下并行應用的調(diào)度算法[J];計算機研究與發(fā)展;2008年04期

3 劉洪偉;于炯;田國忠;龔紅翠;;基于網(wǎng)格資源預測的任務優(yōu)先級調(diào)度算法[J];計算機工程;2009年17期

4 婁淵勝;黃龍達;許洪濤;;Java反射機制在基于O/R映射的數(shù)據(jù)持久層框架中的應用研究[J];計算機應用與軟件;2008年10期

5 楊鐵軍;黃琳;;MINA網(wǎng)絡框架和RMI的對比研究[J];計算機應用與軟件;2010年09期

6 黃橙藍;張灝龍;曹健;;基于BOINC的仿真網(wǎng)格設計與實現(xiàn)[J];計算機應用與軟件;2011年03期

7 謝桂蘭;羅省賢;;基于Hadoop MapReduce模型的應用研究[J];微型機與應用;2010年08期

8 封瑋,周世平;基于Java NIO的非阻塞通信的研究與實現(xiàn)[J];計算機系統(tǒng)應用;2004年09期

9 胡_g;封俊;;Hadoop下的分布式搜索引擎[J];計算機系統(tǒng)應用;2010年07期

10 金玉軍;姜曉彤;況迎輝;;基于高性能NIO的MINA框架的應用[J];中國科技信息;2009年16期

本文編號：1985595