基于不確定理論的云數(shù)據(jù)處理關(guān)鍵技術(shù)研究
發(fā)布時間：2019-06-18 18:05
【摘要】：2016年1月,RightScale對全球1000多個企業(yè)用戶進行了關(guān)于公有云、私有云和混合云的使用情況調(diào)查,該調(diào)查報告結(jié)果顯示95%的受訪者正在使用云�，F(xiàn)實世界中,不確定因素普遍存在于各種現(xiàn)象中。在云計算環(huán)境下,云數(shù)據(jù)中心中云數(shù)據(jù)、虛擬機的遷移、調(diào)度等問題都具有不確定性。對于不確定性數(shù)據(jù)處理,目前已有很多成果,多集中在實體數(shù)據(jù)的不確定性,對現(xiàn)實中一些實際問題覆蓋還不夠。對于實體間關(guān)系的非確定性處理,已有文獻運用隨機和模糊理論解決近鄰查詢處理問題。而實體間關(guān)系有時還表現(xiàn)為主觀不確定性,這種主觀不確定性既不是隨機的也不是模糊的�，F(xiàn)實中,很多問題無法獲得歷史數(shù)據(jù),從而無法用概率論求解事件發(fā)生的頻率,此時必須依據(jù)專家經(jīng)驗對事件可能發(fā)生的信度進行評估,此方法使得信度的方差遠遠大于頻率。為了處理云數(shù)據(jù)的主觀不確定性,將采用不確定理論對云數(shù)據(jù)的處理技術(shù)展開研究。本文致力于云數(shù)據(jù)查詢處理、查詢優(yōu)化關(guān)鍵技術(shù)的研究,由于異構(gòu)性、隱私性、隱私保護、數(shù)據(jù)不完整、數(shù)據(jù)不精確等原因,云數(shù)據(jù)中心的數(shù)據(jù)存在不確定性,借鑒和吸收不確定理論的相關(guān)研究,將云數(shù)據(jù)中心抽象為不確定圖。根據(jù)不確定圖的路徑查詢算法,對云數(shù)據(jù)的查詢處理、查詢優(yōu)化進行深入的探討,本文的主要工作和貢獻可以歸納為：(1)提出了云數(shù)據(jù)安全防護框架。該框架主要包括物理安全、虛擬網(wǎng)絡(luò)安全、云操作系統(tǒng)安全、虛擬集群安全、數(shù)據(jù)安全、SaaS/PaaS/IaaS安全、安全管理與安全運維等層次模塊。該框架在安全目標(biāo)、系統(tǒng)資源類型、基礎(chǔ)安全技術(shù)方面與傳統(tǒng)安全是相同的,而又有其特有的安全問題,主要包括：虛擬化安全問題和與云計算分租服務(wù)模式相關(guān)的一些安全問題。該框架在虛擬化安全、數(shù)據(jù)安全和隱私保護等方面具有更好的安全性和保護能力。(2)提出了基于云數(shù)據(jù)安全防護框架的不確定隨機故障樹風(fēng)險分析方法。該方法基于不確定理論和機會理論,對故障樹進行構(gòu)建和分析。故障樹由基于底事件之間的邏輯關(guān)系構(gòu)成。若底事件的故障率由歷史數(shù)據(jù)獲得,則被表征為隨機變量：若沒有歷史數(shù)據(jù),但可從專家主觀判斷得到,則被表征為不確定變量。除此之外事件發(fā)生的機會是不確定的隨機變量,因此構(gòu)建了混合仿真算法來計算頂事件發(fā)生的機會。通過不確定隨機故障樹分析法對所提出的云數(shù)據(jù)安全防護框架進行風(fēng)險分析。(3)提出了不確定網(wǎng)絡(luò)條件可信近鄰查詢方法。該方法包括可信距離的計算(CMCD)算法,可達路徑長度計算(CMFP)算法,可達路徑期望長度計算(CMDLFP)算法,條件可信k近鄰查詢(QMCCK)算法。將不確定網(wǎng)絡(luò)建模為不確定賦權(quán)圖,定義不確定圖的樣本圖,樣本圖指數(shù),基礎(chǔ)網(wǎng)絡(luò),可達路徑長度及可達路徑期望長度,并給出基于不確定理論的高效不確定條件可信近鄰查詢算法。將不確定網(wǎng)絡(luò)上的近鄰查詢等價地轉(zhuǎn)化為基礎(chǔ)網(wǎng)絡(luò)上的近臨查詢問題。該可信近鄰查詢算法能夠從非確定角度解決不確定網(wǎng)絡(luò)環(huán)境下的近鄰查詢問題。(4)提出了基于不確定理論的不確定性數(shù)據(jù)Top-k查詢算法。將不確定性數(shù)據(jù)集中的元組建模為不確定網(wǎng)絡(luò),將有序元組的Top-k查詢等價轉(zhuǎn)化為相應(yīng)樣本圖中邊的不確定測度關(guān)系,并對樣本圖依據(jù)所包含邊的排序位置進行分類,該算法避免計算所有元組在樣本圖中的排名不確定測度值,提高了不確定性數(shù)據(jù)的Top-k查詢計算效率。將不確定性數(shù)據(jù)中,基于參數(shù)化排名函數(shù)的Top-k查詢等價轉(zhuǎn)換為依Top-k值不同的有限查詢,并結(jié)合Spark Map-Reduce編程框架完成了系統(tǒng)實現(xiàn)。
[Abstract]:In January 2016, the RightScale conducted a survey of the use of public clouds, private clouds and hybrid clouds for more than 1,000 enterprise users worldwide, and the survey found that 95% of the respondents were using the cloud. In the real world, uncertainty is common in various phenomena. In the cloud computing environment, the cloud data in the cloud data center, the migration and the scheduling of the virtual machine and the like have the uncertainty. There are many achievements in the data processing of the uncertainty, and the uncertainty of the entity's data is not enough to cover some of the real problems in the real world. For the non-deterministic processing of the relation between the entities, the existing literature uses the random and fuzzy theory to solve the problem of neighbor query processing. The relationship between the entities is sometimes also subjective, and the subjective uncertainty is neither random nor fuzzy. In reality, many problems can't get the historical data, so we can not use the probability theory to solve the frequency of the event. At this time, it is necessary to evaluate the reliability of the event based on the experience of the experts, which makes the variance of the reliability far greater than the frequency. In order to deal with the subjective uncertainty of cloud data, the process technology of cloud data will be studied with the uncertainty theory. This paper is devoted to the research of the key technology of cloud data query processing and query optimization. Because of the heterogeneity, privacy, privacy protection, incomplete data and inaccurate data, the data of the cloud data center is uncertain, and the relevant research of the uncertainty theory is used for reference and absorption. The cloud data center is abstracted as an uncertainty diagram. According to the path query algorithm of the uncertain graph, the query processing and query optimization of the cloud data are discussed in-depth. The main work and contribution of this paper can be summarized as follows: (1) The cloud data safety protection framework is proposed. The framework mainly includes physical security, virtual network security, cloud operating system security, virtual cluster security, data security, SaaS/ PaaS/ IaaS security, security management and security operation and maintenance level modules. The framework is the same as the traditional security in the aspects of security objective, system resource type and basic security technology, but also has the special security problem, mainly including: the virtualization security problem and some safety problems related to the cloud computing sublease service mode. The framework has better security and protection capabilities in terms of virtualization security, data security, and privacy protection. (2) The risk analysis method for uncertain random fault tree based on cloud data security protection framework is presented. The method is based on the theory of uncertain theory and opportunity, and the fault tree is constructed and analyzed. The fault tree is composed of a logical relationship based on the bottom event. If the failure rate of the bottom event is obtained from the historical data, it is characterized as a random variable: if there is no historical data, it can be obtained from the subjective judgment of the expert and is characterized as an uncertain variable. In addition, the chance of the occurrence of the event is an uncertain random variable, so a hybrid simulation algorithm is constructed to calculate the opportunity for the top event to occur. The proposed cloud data safety protection framework is analyzed by uncertain stochastic fault tree analysis. And (3) the method for querying the trusted neighbor of the network condition is proposed. The method comprises a CMDCD algorithm, a reachable path length calculation (CMDFP) algorithm, a reachable path expectation length calculation (CMDLFP) algorithm, and a conditional trusted k-neighbor query (QMCCK) algorithm. The uncertain network is modeled as an uncertain weight graph, a sample graph, a sample map index, a basic network, a reachable path length and a reachable path expectation length of the uncertain graph are defined, and an efficient and uncertain conditional trusted neighbor query algorithm based on the uncertainty theory is given. The neighbor query on the network is not determined to be equivalently converted into a near-access query problem on the base network. The trusted neighbor query algorithm can solve the problem of neighbor query in the uncertain network environment from the non-deterministic point of view. (4) An uncertain data Top-k query algorithm based on uncertain theory is proposed. the meta-establishment model in the uncertainty data set is a non-deterministic network, and the top-k query of the ordered tuple is equivalent to the uncertainty measure relation of the edge in the corresponding sample graph, and the sample graph is classified according to the sorting position of the included edge, The algorithm avoids the calculation of the uncertainty measure value of all the tuples in the sample graph, and improves the top-k query calculation efficiency of the uncertainty data. In the uncertain data, the top-k query based on the parameterized ranking function is equivalent to a limited query different according to the Top-k value, and the system implementation is completed in combination with the Spark Map-Reduce programming framework.
【學(xué)位授予單位】：北京科技大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2016
【分類號】：TP309
本文編號：2501680
[Abstract]:In January 2016, the RightScale conducted a survey of the use of public clouds, private clouds and hybrid clouds for more than 1,000 enterprise users worldwide, and the survey found that 95% of the respondents were using the cloud. In the real world, uncertainty is common in various phenomena. In the cloud computing environment, the cloud data in the cloud data center, the migration and the scheduling of the virtual machine and the like have the uncertainty. There are many achievements in the data processing of the uncertainty, and the uncertainty of the entity's data is not enough to cover some of the real problems in the real world. For the non-deterministic processing of the relation between the entities, the existing literature uses the random and fuzzy theory to solve the problem of neighbor query processing. The relationship between the entities is sometimes also subjective, and the subjective uncertainty is neither random nor fuzzy. In reality, many problems can't get the historical data, so we can not use the probability theory to solve the frequency of the event. At this time, it is necessary to evaluate the reliability of the event based on the experience of the experts, which makes the variance of the reliability far greater than the frequency. In order to deal with the subjective uncertainty of cloud data, the process technology of cloud data will be studied with the uncertainty theory. This paper is devoted to the research of the key technology of cloud data query processing and query optimization. Because of the heterogeneity, privacy, privacy protection, incomplete data and inaccurate data, the data of the cloud data center is uncertain, and the relevant research of the uncertainty theory is used for reference and absorption. The cloud data center is abstracted as an uncertainty diagram. According to the path query algorithm of the uncertain graph, the query processing and query optimization of the cloud data are discussed in-depth. The main work and contribution of this paper can be summarized as follows: (1) The cloud data safety protection framework is proposed. The framework mainly includes physical security, virtual network security, cloud operating system security, virtual cluster security, data security, SaaS/ PaaS/ IaaS security, security management and security operation and maintenance level modules. The framework is the same as the traditional security in the aspects of security objective, system resource type and basic security technology, but also has the special security problem, mainly including: the virtualization security problem and some safety problems related to the cloud computing sublease service mode. The framework has better security and protection capabilities in terms of virtualization security, data security, and privacy protection. (2) The risk analysis method for uncertain random fault tree based on cloud data security protection framework is presented. The method is based on the theory of uncertain theory and opportunity, and the fault tree is constructed and analyzed. The fault tree is composed of a logical relationship based on the bottom event. If the failure rate of the bottom event is obtained from the historical data, it is characterized as a random variable: if there is no historical data, it can be obtained from the subjective judgment of the expert and is characterized as an uncertain variable. In addition, the chance of the occurrence of the event is an uncertain random variable, so a hybrid simulation algorithm is constructed to calculate the opportunity for the top event to occur. The proposed cloud data safety protection framework is analyzed by uncertain stochastic fault tree analysis. And (3) the method for querying the trusted neighbor of the network condition is proposed. The method comprises a CMDCD algorithm, a reachable path length calculation (CMDFP) algorithm, a reachable path expectation length calculation (CMDLFP) algorithm, and a conditional trusted k-neighbor query (QMCCK) algorithm. The uncertain network is modeled as an uncertain weight graph, a sample graph, a sample map index, a basic network, a reachable path length and a reachable path expectation length of the uncertain graph are defined, and an efficient and uncertain conditional trusted neighbor query algorithm based on the uncertainty theory is given. The neighbor query on the network is not determined to be equivalently converted into a near-access query problem on the base network. The trusted neighbor query algorithm can solve the problem of neighbor query in the uncertain network environment from the non-deterministic point of view. (4) An uncertain data Top-k query algorithm based on uncertain theory is proposed. the meta-establishment model in the uncertainty data set is a non-deterministic network, and the top-k query of the ordered tuple is equivalent to the uncertainty measure relation of the edge in the corresponding sample graph, and the sample graph is classified according to the sorting position of the included edge, The algorithm avoids the calculation of the uncertainty measure value of all the tuples in the sample graph, and improves the top-k query calculation efficiency of the uncertainty data. In the uncertain data, the top-k query based on the parameterized ranking function is equivalent to a limited query different according to the Top-k value, and the system implementation is completed in combination with the Spark Map-Reduce programming framework.
【學(xué)位授予單位】：北京科技大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2016
【分類號】：TP309
【相似文獻】
相關(guān)期刊論文 前10條
1 ;連續(xù)性數(shù)據(jù)分組怎樣表示最科學(xué)?[J];無錫職業(yè)技術(shù)學(xué)院學(xué)報;2012年04期
2 葛楊;徐名海;遲歡;;關(guān)于傳輸虛擬化中數(shù)據(jù)分組亂序問題的研究[J];電信科學(xué);2012年10期
3 葉玉杰;邱丘;陳亞軍;張永忠;劉嵩;胡霞敏;;運用Origin軟件處理藥學(xué)實驗數(shù)據(jù)[J];藥學(xué)實踐雜志;2013年06期
4 董永吉;郭云飛;黃萬偉;黃慧群;;面向深度分組檢測的高速數(shù)據(jù)分組解析結(jié)構(gòu)[J];通信學(xué)報;2013年06期
5 高衛(wèi)民;數(shù)據(jù)處理組合軟件的原理與技術(shù)[J];計算機應(yīng)用研究;1991年02期
6 王銳;陳麗;馬方明;;一種電信行業(yè)海量數(shù)據(jù)分組統(tǒng)計方法[J];計算機應(yīng)用與軟件;2012年12期
7 ;艾法斯推出業(yè)界最真實的LTE服務(wù)測試流量場景[J];移動通信;2012年24期
8 李玉海;田苗苗;黃劉生;楊威;;無線傳感網(wǎng)絡(luò)中基于數(shù)據(jù)混淆的保護隱私數(shù)據(jù)聚集協(xié)議[J];小型微型計算機系統(tǒng);2013年07期
9 李霽;麻土華;;基于W函數(shù)的數(shù)據(jù)分組方法的算法實現(xiàn)[J];科技通報;2012年05期
10 鄭海鷗,張乃通;無線數(shù)據(jù)擴頻MODEM組網(wǎng)[J];無線電通信技術(shù);2000年03期
相關(guān)重要報紙文章 前1條
1 本報記者 趙姍;大數(shù)據(jù)時代來臨，中國準(zhǔn)備好了嗎？[N];中國經(jīng)濟時報;2013年
相關(guān)博士學(xué)位論文 前3條
1 劉海青;大規(guī)模VANET數(shù)據(jù)傳輸策略的研究[D];山東大學(xué);2015年
2 郭長友;基于不確定理論的云數(shù)據(jù)處理關(guān)鍵技術(shù)研究[D];北京科技大學(xué);2016年
3 劉琴;多用戶共享云計算服務(wù)環(huán)境下安全問題研究[D];中南大學(xué);2012年
相關(guān)碩士學(xué)位論文 前10條
1 張忠賀;Modbus/TCP協(xié)議在WIFI應(yīng)用通訊下的實現(xiàn)[D];內(nèi)蒙古大學(xué);2015年
2 喬新生;量子密鑰分發(fā)系統(tǒng)中數(shù)據(jù)協(xié)調(diào)算法及軟件設(shè)計[D];西安電子科技大學(xué);2014年
3 楊建東;云環(huán)境下網(wǎng)管數(shù)據(jù)查詢系統(tǒng)設(shè)計[D];南京郵電大學(xué);2015年
4 汪鵬飛;異構(gòu)網(wǎng)絡(luò)中基于MPTCP多路傳輸?shù)臄?shù)據(jù)調(diào)度策略研究[D];華中師范大學(xué);2015年
5 劉娟;橋梁健康監(jiān)測系統(tǒng)的設(shè)計與實現(xiàn)[D];電子科技大學(xué);2011年
6 陳娜;基于Hadoop平臺的海量數(shù)據(jù)處理應(yīng)用[D];吉林大學(xué);2012年
7 蔣杰;基于GPRS的嵌入式系統(tǒng)安全無堵塞通信研究與設(shè)計[D];哈爾濱工程大學(xué);2013年
8 蔣穎;面向卷煙制絲線的SPC系統(tǒng)的應(yīng)用研究與實現(xiàn)[D];湖南大學(xué);2012年
9 羅恩澤;面向大規(guī)模工程機械遠程智能監(jiān)控的無線通信協(xié)議[D];湖南大學(xué);2011年
10 付金光;電力系統(tǒng)不良數(shù)據(jù)辨識的實用軟件開發(fā)[D];鄭州大學(xué);2011年
，本文編號：2501680
本文鏈接：http://sikaile.net/shoufeilunwen/xxkjbs/2501680.html
最近更新
教材專著
