本文選題：開放式海量數(shù)據(jù)處理服務(wù) + 計算完整性　； 參考：《國防科學(xué)技術(shù)大學(xué)》2014年博士論文

【摘要】：開放式海量數(shù)據(jù)處理服務(wù)在大數(shù)據(jù)處理中發(fā)揮著越來越重要的作用。然而,由于開放式服務(wù)可能面臨來自服務(wù)方主觀意圖以及內(nèi)部分布式計算環(huán)境客觀安全因素等威脅,如何保證服務(wù)的計算完整性成為重要問題。現(xiàn)有研究主要對海量數(shù)據(jù)處理的內(nèi)部計算框架進(jìn)行研究,通過多副本技術(shù)對計算節(jié)點提供的結(jié)果進(jìn)行檢測,從而保證計算任務(wù)的計算完整性。由于基于多副本的技術(shù)將會帶來很大的計算開銷,因此限制了方法的實際可用性;并且這類針對內(nèi)部計算節(jié)點的檢測機(jī)制并不能有效解決服務(wù)方主觀欺騙造成的計算完整性問題。針對開放式海量數(shù)據(jù)處理服務(wù)計算完整性問題,本文從計算完整性檢測與計算完整性保障兩個方面展開研究。計算完整性檢測問題關(guān)注從用戶角度對服務(wù)方的計算行為以及結(jié)果的完整性檢測,屬于計算完整性的事后檢查;計算完整性保障問題則從服務(wù)方角度研究如何組織可信的計算資源得到滿足計算完整性要求的計算結(jié)果,屬于計算完整性的主動保護(hù)。本文以當(dāng)前海量數(shù)據(jù)處理的主流計算模式Map Reduce為研究對象,結(jié)合海量數(shù)據(jù)處理的計算特點,從提高方法的可用性與計算效能出發(fā),以降低計算完整性檢測機(jī)制的性能開銷、提高計算完整性保障力度為優(yōu)化目標(biāo),系統(tǒng)的研究了開放式海量數(shù)據(jù)處理服務(wù)計算完整性方面的若干重要問題。本文的主要研究內(nèi)容及創(chuàng)新點包括以下幾方面:第一,研究了基于第三方的計算完整性檢測問題。云服務(wù)模式下,建立可控的云計算安全監(jiān)管體系是云服務(wù)可信研究要解決的重要挑戰(zhàn),對服務(wù)進(jìn)行基于第三方的審計是其中的重要手段。在Map Reduce計算機(jī)制中,Map計算為對用戶原始輸入的處理,是計算的重要組成部分。本文提出了“基于第三方可信抽樣的Map階段計算完整性檢測”機(jī)制,通過可信第三方對Map Reduce中間結(jié)果進(jìn)行抽樣檢測,以少量的檢測開銷來檢測服務(wù)方Map階段的計算完整性,并且針對服務(wù)商可能存在的主觀上的不配合問題,利用Merkle樹技術(shù)對檢測結(jié)果進(jìn)行組織,防止服務(wù)方為應(yīng)付審計而進(jìn)行欺騙行為,保證檢測結(jié)果真實可靠。第二,研究了計算完整性用戶自主檢測問題。在云服務(wù)監(jiān)管體系的建立完善之前,研究服務(wù)方不感知的用戶自主檢測方法也成為解決問題的有效途徑。本文研究了“基于監(jiān)控探針的計算完整性用戶自主檢測方法”,根據(jù)Map Reduce計算問題的類型構(gòu)建預(yù)先知道計算結(jié)果的監(jiān)控探針并注入到輸入數(shù)據(jù)集合中,通過探針數(shù)據(jù)的計算結(jié)果來以一定概率檢測整體計算任務(wù)是否滿足計算完整性要求。由于該方法與具體的計算類型相關(guān),因此,本文重點對該方法進(jìn)行建模,對其重要性質(zhì)進(jìn)行研究,并針對幾種典型的Map Reduce計算類型研究監(jiān)控探針的構(gòu)造方法。該方法可以對Map與Reduce的全部計算階段進(jìn)行完整性檢測,并且無需服務(wù)方的配合即可得出檢測結(jié)果;同時由于該方法基于抽樣檢測思想,檢測的開銷可以被用戶所接受。第三,研究了可信開放式Map Reduce系統(tǒng)構(gòu)建問題。在服務(wù)方內(nèi)部,當(dāng)服務(wù)方利用開放式計算資源來組織計算系統(tǒng)時,由于計算資源可能來自不同的可信域,因此,需要對各節(jié)點的計算結(jié)果進(jìn)行檢測,只有通過檢測的結(jié)果才能被采納。當(dāng)前的檢測手段主要通過多副本檢測,而對于多副本機(jī)制在抵御共謀攻擊方面的弱點,則缺乏高效的解決手段。本文提出了“抗共謀的開放式環(huán)境下可信Map Reduce系統(tǒng)構(gòu)建”方法,該方法無需針對共謀攻擊設(shè)計額外的檢測機(jī)制,僅利用多副本檢測的歷史信息,即可定位共謀與非共謀攻擊模式下的惡意節(jié)點。該方法利用完整性證明圖來描述系統(tǒng)內(nèi)的節(jié)點間多副本檢測關(guān)系,并基于完整性證明圖的最大團(tuán)分析來精確定位惡意節(jié)點。并且,該方法還提出了基于完整性證明圖指導(dǎo)的節(jié)點檢測對選擇啟發(fā)式算法,指導(dǎo)檢測副本對的選擇,提高惡意節(jié)點檢測的效率。第四,研究了計算節(jié)點可信性評估問題。在海量數(shù)據(jù)處理中,盡管多副本技術(shù)的檢測精度高,但是由于參與計算的節(jié)點數(shù)量大,如果完全應(yīng)用多副本技術(shù)來進(jìn)行檢測,會導(dǎo)致巨大的計算開銷。如果能夠利用很小的檢測成本,對節(jié)點的可信性進(jìn)行預(yù)評估,然后在此基礎(chǔ)上,在實際運行的系統(tǒng)中針對可信度低的節(jié)點再應(yīng)用多副本檢測,將大大提高檢測機(jī)制的計算效率。本文提出了“基于監(jiān)控探針的計算節(jié)點可信性評估方法”,通過探針數(shù)據(jù)的計算結(jié)果判斷探針在系統(tǒng)中是否被正確執(zhí)行,并結(jié)合Map Reduce的Shuffle機(jī)制,確定監(jiān)控探針的執(zhí)行路徑,通過信譽機(jī)制對各參與計算的節(jié)點進(jìn)行可信性評估。該方法工作在應(yīng)用級,無需對計算框架進(jìn)行修改。而通過對可信節(jié)點的評估,可以對那些低可信性排名的節(jié)點進(jìn)行精度更高的多副本檢測,從而有效降低檢測機(jī)制的計算資源需求。綜上所述,本文對開放式海量數(shù)據(jù)處理服務(wù)的計算完整性問題進(jìn)行了深入的研究,提出了具有高可用性、高檢測率、低開銷的解決方案,并通過理論分析和大量的實驗驗證了所提出方法的有效性和性能,對于建立誠實可信的開放式海量數(shù)據(jù)處理服務(wù)環(huán)境具有一定的理論意義和應(yīng)用價值。
[Abstract]:Open mass data processing services play a more and more important role in large data processing. However, because open services may face threats from the subjective intention of the service parties and the objective security factors in the internal distributed computing environment, how to ensure the integrity of the service has become an important problem. The internal computing framework of data processing is studied to detect the results of computing nodes by multi replica technology to ensure the computational integrity of computing tasks. Because the multi replica based technology will bring a lot of computing overhead, the practical availability of the method is limited; and this kind of inspection for internal computing nodes. The measurement mechanism can not effectively solve the computing integrity problem caused by the subjective deception of the service party. Aiming at the integrity problem of the open mass data processing service, this paper studies the two aspects of the integrity detection and the security of computing integrity. And the integrity detection of the results, it belongs to the ex post examination of computing integrity; the problem of computing integrity protection is to study how to organize the trusted computing resources to meet the computing integrity requirements from the point of view of the service side, which belongs to the active protection of computing integrity. This paper is based on the mainstream computing model Map of the current mass data processing. Reduce is the research object. Combining with the computing characteristics of mass data processing, starting from improving the availability and computing efficiency of the method, it reduces the performance overhead of the computing integrity detection mechanism and improves the strength of computing integrity as an optimization goal. The system has studied several important aspects of the integrity of open mass data processing services. The main research contents and innovation points of this paper include the following aspects: first, it studies the problem of computing integrity detection based on third parties. Under the cloud service mode, the establishment of a controlled cloud computing security supervision system is an important challenge to solve the cloud service trust research. The audit of the service based on the third party is important. Means. In the Map Reduce computer system, Map is an important part of the computation for the processing of the original input to the user. This paper proposes a "Map phase computation integrity detection based on third party trusted sampling", which is sampled by a trusted third party to the intermediate result of Map Reduce, and detects the clothing with a small amount of detection overhead. The computing integrity of the Map phase, and the possible subjective non coordination of the service providers, using the Merkle tree technology to organize the detection results, to prevent the service parties from cheating on the audit, and to ensure the true and reliable results of the detection. Second, the problem of independent testing of the computing integrity users is studied. Before the establishment and perfection of the supervision system, the method of user independent detection which is not perceived by the service party is also an effective way to solve the problem. This paper studies the method of "computing integrity user independent detection based on monitoring probe". According to the type of Map Reduce computing problem, the monitoring probe that pre know the results is built and injected into the monitoring system. In the input data set, the calculation results of the probe data are used to determine whether the overall computing task is satisfied with the computing integrity requirements. Because the method is related to the specific type of calculation, this paper focuses on the modeling of the method, studies its important properties, and aims at several typical Map Reduce computing types. Study the construction method of monitoring probe. This method can carry out integrity detection for all calculation stages of Map and Reduce, and can get the detection results without the need of the cooperation of the service side. At the same time, the detection cost can be accepted by the user because of the method based on sampling detection. Third, the construction of the trusted open Map Reduce system is studied. In the service side, when the service party organizes the computing system with open computing resources, the computing resource may come from different trusted domains. Therefore, it needs to detect the results of each node, only through the result of detection. The replica mechanism is lack of efficient solution to the weakness of conspiracy attack. This paper proposes a "trusted Map Reduce system construction under an anti conspiracy open environment". This method does not need to design an additional detection mechanism for conspiracy attack and only uses the historical information of multi copy detection to locate conspiracy and non conspiracy. A malicious node under attack mode. This method uses integrity proof graph to describe the multiple copy detection relationship among nodes in the system, and accurately locates the malicious nodes based on the maximum group analysis of the integrity proof graph. Furthermore, this method also proposes a heuristic algorithm based on the integrity proof diagram to guide the detection of the selection. The selection of copy pairs improves the efficiency of detection of malicious nodes. Fourth, the reliability evaluation of computing nodes is studied. In the process of massive data processing, although the detection precision of multi copy technology is high, the number of nodes involved in the computation is large, and if the multiple copy technology is used to detect it, it will lead to huge computational overhead. It can make use of small detection cost and pre evaluate the credibility of nodes. Then, using multi copy detection for low reliability nodes in the actual running system, it will greatly improve the computing efficiency of the detection mechanism. This paper proposes a method for evaluating the credibility of the computing nodes based on the monitoring probe. The calculation results of the probe data determine whether the probe is properly executed in the system, and combines the Shuffle mechanism of the Map Reduce to determine the execution path of the monitoring probe. The credibility mechanism is used to evaluate the credibility of the nodes involved in the calculation. In this paper, the computational integrity of the open mass data processing service is deeply studied, and the solutions with high availability, high detection rate and low overhead are proposed. Through theoretical analysis and a large number of experiments, the effectiveness and performance of the proposed method are verified. It has a certain theoretical significance and application value for the establishment of an honest and credible open mass data processing service environment.

【學(xué)位授予單位】：國防科學(xué)技術(shù)大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2014
【分類號】：TP393.08

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 高倩;王慧勇;張悠慧;汪東升;;基于瘦客戶的混合計算設(shè)計與實現(xiàn)[J];小型微型計算機(jī)系統(tǒng);2007年06期

2 李雪冬;;云計算及其應(yīng)用[J];科技信息;2011年24期

3 李育林;;云計算的局限性與解決方案[J];黃岡師范學(xué)院學(xué)報;2011年06期

4 陳瓊;公用計算:動靜兩相宜?[J];互聯(lián)網(wǎng)周刊;2003年41期

5 武星;王e，

本文編號：1784350