本文選題：分布式 + 計算框架�。� 參考：《華中科技大學(xué)》2012年碩士論文

【摘要】：近年來隨著計算機(jī)網(wǎng)絡(luò)技術(shù)的高速發(fā)展，互聯(lián)網(wǎng)企業(yè)為用戶提供的應(yīng)用和個性化服務(wù)也越來越多，隨之而來的是用戶提交的海量數(shù)據(jù)，企業(yè)必須及時而又正確地處理這些數(shù)據(jù)并向用戶反饋結(jié)果。通常，人們會選擇使用分布式計算系統(tǒng)來完成這些復(fù)雜的任務(wù)。然而傳統(tǒng)的分布式計算平臺往往需要配置大型的服務(wù)器集群，還需要專門的技術(shù)人員去跟進(jìn)和維護(hù)，這給很多企業(yè)都帶來了巨大的經(jīng)濟(jì)開銷。 輕量級分布式任務(wù)計算框架Antnest采用了客戶機(jī)/服務(wù)器計算模式和三層體系結(jié)構(gòu)，可以將工作任務(wù)分布在廉價的任務(wù)節(jié)點(diǎn)上，中心節(jié)點(diǎn)負(fù)責(zé)整個系統(tǒng)的調(diào)度工作，任務(wù)節(jié)點(diǎn)采用多JVM多線程執(zhí)行任務(wù)。在Antnest中，提交的任務(wù)都是經(jīng)過定制的，不同的任務(wù)對實(shí)時性的要求各不相同，針對此，Antnest為任務(wù)調(diào)度系統(tǒng)設(shè)計了一種多級隊(duì)列搶占式調(diào)度算法，該算法充分考慮到任務(wù)對實(shí)時性的要求，高優(yōu)先級任務(wù)可以搶占系統(tǒng)，，被搶占任務(wù)會移入高優(yōu)先級隊(duì)列中等待調(diào)度，同時利用任務(wù)的預(yù)定啟動時間來確保任務(wù)執(zhí)行的有效性。另外，Antnest還設(shè)計了動態(tài)任務(wù)分配算法來完成子任務(wù)的分配工作。在動態(tài)分配算法中，子任務(wù)和任務(wù)節(jié)點(diǎn)的分配關(guān)系是在任務(wù)的執(zhí)行過程中動態(tài)確定的，Antnest會優(yōu)先將子任務(wù)分配給最先執(zhí)行完子任務(wù)的任務(wù)節(jié)點(diǎn)。在任務(wù)調(diào)度系統(tǒng)的實(shí)現(xiàn)中，任務(wù)調(diào)度系統(tǒng)各模塊間通過自定義的通信協(xié)議完成消息通信，通過指定的任務(wù)接口觸發(fā)任務(wù)邏輯，利用任務(wù)節(jié)點(diǎn)的狀態(tài)管理機(jī)制完成資源管理并保證Antnest的可擴(kuò)展性，同時，利用心跳檢測和子任務(wù)重傳的容錯機(jī)制確保任務(wù)的正確執(zhí)行。 測試結(jié)果表明，Antnest系統(tǒng)能有效地提高任務(wù)的執(zhí)行效率，減短執(zhí)行時間；同時，結(jié)果也驗(yàn)證了Antnest系統(tǒng)中多級隊(duì)列搶占式調(diào)度算法的合理性和可行性。
[Abstract]:In recent years, with the rapid development of computer network technology, Internet enterprises provide users with more and more applications and personalized services, followed by massive data submitted by users. The enterprise must process the data in a timely and correct manner and feedback the results to the user. Usually, people choose to use distributed computing systems to accomplish these complex tasks. However, traditional distributed computing platforms often need to configure large server clusters, and also need specialized technicians to follow up and maintain, which brings huge economic expenses to many enterprises. The lightweight distributed task computing framework (Antnest) adopts the client / server computing model and three-tier architecture, which can distribute the work tasks on the cheap task nodes, and the central node is responsible for the scheduling of the whole system. The task node uses multi-JVM multi-thread to execute the task. In Antnest, all the submitted tasks are customized, and different tasks require different real-time performance. A multi-level queue preemptive scheduling algorithm is designed for this task scheduling system. The algorithm fully takes into account the real-time requirements of tasks, high priority tasks can preemptive system, preemptive tasks will be moved into high priority queue to wait for scheduling, and the scheduled start time of tasks will be used to ensure the effectiveness of task execution. In addition, Antnest designs a dynamic task allocation algorithm to accomplish the task assignment. In the dynamic assignment algorithm, the relationship between subtasks and task nodes is dynamically determined in the process of task execution. In the implementation of task scheduling system, message communication is accomplished between modules of task scheduling system through self-defined communication protocol, and task logic is triggered by specified task interface. The state management mechanism of the task node is used to accomplish resource management and to ensure the scalability of the Antnest. At the same time, the fault-tolerant mechanism of heartbeat detection and sub-task retransmission is used to ensure the correct execution of the task. The test results show that the Antnest system can effectively improve the task execution efficiency and reduce the execution time, and it also verifies the rationality and feasibility of the multilevel queue preemptive scheduling algorithm in Antnest system.
【學(xué)位授予單位】：華中科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2012
【分類號】：TP338.8

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 李本亮;王厚軍;李力;;基于WLAN的分布式測試系統(tǒng)實(shí)時性研究[J];電子測量與儀器學(xué)報;2008年06期

2 趙明宇;張?zhí)镂?;一種分布式計算環(huán)境下并行應(yīng)用的調(diào)度算法[J];計算機(jī)研究與發(fā)展;2008年04期

3 劉洪偉;于炯;田國忠;龔紅翠;;基于網(wǎng)格資源預(yù)測的任務(wù)優(yōu)先級調(diào)度算法[J];計算機(jī)工程;2009年17期

4 婁淵勝;黃龍達(dá);許洪濤;;Java反射機(jī)制在基于O/R映射的數(shù)據(jù)持久層框架中的應(yīng)用研究[J];計算機(jī)應(yīng)用與軟件;2008年10期

5 楊鐵軍;黃琳;;MINA網(wǎng)絡(luò)框架和RMI的對比研究[J];計算機(jī)應(yīng)用與軟件;2010年09期

6 黃橙藍(lán);張灝龍;曹健;;基于BOINC的仿真網(wǎng)格設(shè)計與實(shí)現(xiàn)[J];計算機(jī)應(yīng)用與軟件;2011年03期

7 謝桂蘭;羅省賢;;基于Hadoop MapReduce模型的應(yīng)用研究[J];微型機(jī)與應(yīng)用;2010年08期

8 封瑋,周世平;基于Java NIO的非阻塞通信的研究與實(shí)現(xiàn)[J];計算機(jī)系統(tǒng)應(yīng)用;2004年09期

9 胡_g;封俊;;Hadoop下的分布式搜索引擎[J];計算機(jī)系統(tǒng)應(yīng)用;2010年07期

10 金玉軍;姜曉彤;況迎輝;;基于高性能NIO的MINA框架的應(yīng)用[J];中國科技信息;2009年16期

本文編號：1985596