基于云計(jì)算的神經(jīng)網(wǎng)絡(luò)并行實(shí)現(xiàn)及其學(xué)習(xí)方法研究
發(fā)布時間：2019-05-21 23:22
【摘要】：隨著網(wǎng)絡(luò)技術(shù)和軟件技術(shù)及云計(jì)算技術(shù)的高速發(fā)展,當(dāng)前數(shù)據(jù)正以海量的方式遞增,并已經(jīng)進(jìn)入了大數(shù)據(jù)時代。真實(shí)世界數(shù)據(jù),比如數(shù)碼照片、基因表達(dá)譜、人臉數(shù)據(jù)集或網(wǎng)頁文本,通常具有維數(shù)高和數(shù)據(jù)量大的特點(diǎn)。對于傳統(tǒng)的人工智能技術(shù)和模式識別技術(shù)等都面臨如何在大數(shù)據(jù)時代下實(shí)現(xiàn)數(shù)據(jù)處理的挑戰(zhàn)。比如,對于大規(guī)模的人臉數(shù)據(jù)集分類,一臺計(jì)算機(jī)或工作站因?yàn)槿狈λ俣群痛鎯θ萘亢茈y適應(yīng)實(shí)際需求。為此,非常有必要研究在大數(shù)據(jù)環(huán)境下如何實(shí)現(xiàn)基于多計(jì)算機(jī)集群的人工智能技術(shù)和模式識別技術(shù)。當(dāng)采用人工智能方法,比如利用神經(jīng)網(wǎng)絡(luò)對相關(guān)數(shù)據(jù)進(jìn)行處理時,若訓(xùn)練樣本的數(shù)量規(guī)模不大時,單個神經(jīng)網(wǎng)絡(luò)的泛化能力和運(yùn)行時間是比較理想的。然而隨著識別類別及數(shù)目增加,神經(jīng)網(wǎng)絡(luò)的結(jié)構(gòu)也將變得更加復(fù)雜,導(dǎo)致神經(jīng)網(wǎng)絡(luò)訓(xùn)練時間變得更長,收斂速度變得更慢,容易陷入局部最小值和更差的泛化能力等。為了解決這些問題,本論文研究和設(shè)計(jì)了由多個神經(jīng)網(wǎng)絡(luò)組成的集成神經(jīng)網(wǎng)絡(luò)(Hybrid Neural Networks,HNNs)去代替復(fù)雜的單一神經(jīng)網(wǎng)絡(luò),并且提出了一種新穎的半監(jiān)督學(xué)習(xí)算法——嵌入Softmax回歸的深度信念網(wǎng)絡(luò)(Deep Belief Network Embedded with Softmax Regress,DBNESR)作為分類器的深度學(xué)習(xí)方法。本論文所做的主要貢獻(xiàn)如下:(1)本文提出了一種在云計(jì)算集群上,基于Map-Reduce的多層神經(jīng)網(wǎng)絡(luò)并行實(shí)現(xiàn)方法。也即為了滿足大數(shù)據(jù)處理的需要,本文提出了一種在云計(jì)算集群上,基于Map-Reduce的誤差反傳BP算法被訓(xùn)練的全連接多層神經(jīng)網(wǎng)絡(luò)的有效映射機(jī)制。針對一個在云計(jì)算集群上的并行BP算法和一個在單一處理機(jī)上的串行BP算法,從理論上推導(dǎo)了實(shí)現(xiàn)算法所需要的時間,并且評估了在云計(jì)算集群上的并行BP算法及性能參數(shù)(加速比、數(shù)據(jù)節(jié)點(diǎn)的最佳數(shù)目和最小數(shù)目等)。實(shí)驗(yàn)結(jié)果證明,本文提出的并行BP算法比現(xiàn)有的算法有更好的加速比和更快的收斂速率及更少的迭代次數(shù)。(2)本文提出了一種在云計(jì)算集群上,基于Map-Reduce的徑向基函數(shù)神經(jīng)網(wǎng)絡(luò)的并行實(shí)現(xiàn)方法,并進(jìn)行了情感計(jì)算等應(yīng)用研究。也即借助于云計(jì)算平臺,通過網(wǎng)絡(luò)流通和組合提供的計(jì)算能力,實(shí)現(xiàn)了徑向基函數(shù)神經(jīng)網(wǎng)絡(luò)及學(xué)習(xí)算法的并行訓(xùn)練和分類識別應(yīng)用,從而使徑向基函數(shù)神經(jīng)網(wǎng)絡(luò)能夠進(jìn)行跨平臺的學(xué)習(xí),以及處理人臉識別和語音識別及情感計(jì)算等海量的高維數(shù)據(jù)。實(shí)驗(yàn)結(jié)果表明,本文提出的算法比基于單一計(jì)算機(jī)的傳統(tǒng)串行訓(xùn)練神經(jīng)網(wǎng)絡(luò)學(xué)習(xí)算法有更快的學(xué)習(xí)速度,更高的識別率,更大的數(shù)據(jù)處理能力。(3)本文提出了一種半監(jiān)督學(xué)習(xí)算法——內(nèi)嵌Softmax回歸的深度信念網(wǎng)絡(luò)(DBNESR),并且設(shè)計(jì)了多種基于監(jiān)督學(xué)習(xí)的分類器:BP、HBPNNs、RBF、HRBFNNs、SVM、多分類決策融合分類器(Multiple Classification Decision Fusion Classifier,MCDFC)——集成HBPNNs-HRBFNNs-SVM分類器。實(shí)驗(yàn)結(jié)果表明,半監(jiān)督深度算法DBNESR具有較佳的、較高、較穩(wěn)定的識別率;半監(jiān)督學(xué)習(xí)算法比所有的監(jiān)督學(xué)習(xí)算法有更好的效果;集成神經(jīng)網(wǎng)絡(luò)比單一神經(jīng)網(wǎng)絡(luò)有更好的效果;平均識別率和方差分別為BPHBPNNs≈RBFHRBFNNs≈SVMMCDFCDBNESR和BPRBFHBPNNsHRBFNNsSVMMCDFCDBNESR;這反映了DBNESR具有模擬復(fù)雜人工智能任務(wù)的能力。
[Abstract]:With the rapid development of network technology and software technology and cloud computing technology, the current data is increasing in a massive way and has entered a large data era. Real-world data, such as digital photographs, gene expression profiles, face data sets, or web pages, typically have the characteristics of high dimensionality and large data volume. The traditional artificial intelligence technology and pattern recognition technology are faced with the challenge of how to realize the data processing in the big data age. For example, for large-scale face data sets, a computer or workstation is difficult to adapt to the actual needs because of a lack of speed and storage capacity. To this end, it is necessary to study how to realize the technology of artificial intelligence and pattern recognition based on the multi-computer cluster in the large data environment. When the number of training samples is not large, the generalization ability and the running time of a single neural network are ideal when the number of training samples is not large when the artificial intelligence method is adopted, such as using the neural network to process the related data. However, with the increase of the identification category and number, the structure of the neural network will become more complex, resulting in the neural network training time becoming longer, the convergence speed becomes slower, the local minimum value and the worse generalization ability can be easily trapped. In order to solve these problems, this paper studies and designs an integrated neural network (HNNs) which is composed of a plurality of neural networks instead of a complex single neural network. A novel semi-supervised learning algorithm, Deep Belly Network Embedded with Softmax Repress (DBESSR), is proposed as the depth learning method of the classifier. The main contribution of this thesis is as follows: (1) This paper presents a multi-layer neural network parallel implementation method based on Map-Reduce on the cloud computing cluster. In order to satisfy the need of large data processing, this paper presents an effective mapping mechanism of a fully connected multi-layer neural network trained on the cloud computing cluster, based on the error back-propagation BP algorithm of Map-Reduce. Aiming at a parallel BP algorithm on a cloud computing cluster and a serial BP algorithm on a single processor, the time required for implementing the algorithm is derived theoretically, and the parallel BP algorithm and the performance parameter (acceleration ratio) on the cloud computing cluster are evaluated, The optimal number and the minimum number of data nodes, etc.). The experimental results show that the proposed parallel BP algorithm has better speedup and faster convergence rate and lower number of iterations than the existing algorithms. (2) In this paper, a parallel realization method of the radial basis function neural network based on Map-Reduce is proposed in the cloud computing cluster. in other words, by means of the computing capability provided by the cloud computing platform and through the network flow and the combination, the parallel training and classification identification applications of the radial basis function neural network and the learning algorithm are realized, so that the radial basis function neural network can carry out cross-platform learning, And processing the massive high-dimensional data such as face recognition and speech recognition and emotion calculation. The experimental results show that the algorithm proposed in this paper has a faster learning speed, higher recognition rate and greater data processing capacity than the traditional serial training neural network learning algorithm based on a single computer. (3) In this paper, a semi-supervised learning algorithm _ embedded Softmax regression depth belief network (DBNESR) is proposed, and a variety of supervised learning-based classifiers are designed: BP, HBPNNs, RBF, HRBFNs, SVMs, multi-classification decision fusion classifier (MCDFC) _ integrated HBPNNs-HRBNs-SVM classifier. The experimental results show that the semi-supervised depth algorithm DBNESR has better, higher and stable recognition rate, and the semi-supervised learning algorithm has better effect than all of the supervised learning algorithms, and the integrated neural network has better effect than the single neural network. The average recognition rate and variance are BPHBPNNs, RBFHRBFNNs, SVMMCDFCDBNESR and BPRBHBPNNsHRBFNNsSVMMCDFCDBNESR, respectively; this reflects the ability of the DBNESR to simulate complex artificial intelligence tasks.
【學(xué)位授予單位】：華南理工大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2015
【分類號】：TP183
本文編號：2482489
[Abstract]:With the rapid development of network technology and software technology and cloud computing technology, the current data is increasing in a massive way and has entered a large data era. Real-world data, such as digital photographs, gene expression profiles, face data sets, or web pages, typically have the characteristics of high dimensionality and large data volume. The traditional artificial intelligence technology and pattern recognition technology are faced with the challenge of how to realize the data processing in the big data age. For example, for large-scale face data sets, a computer or workstation is difficult to adapt to the actual needs because of a lack of speed and storage capacity. To this end, it is necessary to study how to realize the technology of artificial intelligence and pattern recognition based on the multi-computer cluster in the large data environment. When the number of training samples is not large, the generalization ability and the running time of a single neural network are ideal when the number of training samples is not large when the artificial intelligence method is adopted, such as using the neural network to process the related data. However, with the increase of the identification category and number, the structure of the neural network will become more complex, resulting in the neural network training time becoming longer, the convergence speed becomes slower, the local minimum value and the worse generalization ability can be easily trapped. In order to solve these problems, this paper studies and designs an integrated neural network (HNNs) which is composed of a plurality of neural networks instead of a complex single neural network. A novel semi-supervised learning algorithm, Deep Belly Network Embedded with Softmax Repress (DBESSR), is proposed as the depth learning method of the classifier. The main contribution of this thesis is as follows: (1) This paper presents a multi-layer neural network parallel implementation method based on Map-Reduce on the cloud computing cluster. In order to satisfy the need of large data processing, this paper presents an effective mapping mechanism of a fully connected multi-layer neural network trained on the cloud computing cluster, based on the error back-propagation BP algorithm of Map-Reduce. Aiming at a parallel BP algorithm on a cloud computing cluster and a serial BP algorithm on a single processor, the time required for implementing the algorithm is derived theoretically, and the parallel BP algorithm and the performance parameter (acceleration ratio) on the cloud computing cluster are evaluated, The optimal number and the minimum number of data nodes, etc.). The experimental results show that the proposed parallel BP algorithm has better speedup and faster convergence rate and lower number of iterations than the existing algorithms. (2) In this paper, a parallel realization method of the radial basis function neural network based on Map-Reduce is proposed in the cloud computing cluster. in other words, by means of the computing capability provided by the cloud computing platform and through the network flow and the combination, the parallel training and classification identification applications of the radial basis function neural network and the learning algorithm are realized, so that the radial basis function neural network can carry out cross-platform learning, And processing the massive high-dimensional data such as face recognition and speech recognition and emotion calculation. The experimental results show that the algorithm proposed in this paper has a faster learning speed, higher recognition rate and greater data processing capacity than the traditional serial training neural network learning algorithm based on a single computer. (3) In this paper, a semi-supervised learning algorithm _ embedded Softmax regression depth belief network (DBNESR) is proposed, and a variety of supervised learning-based classifiers are designed: BP, HBPNNs, RBF, HRBFNs, SVMs, multi-classification decision fusion classifier (MCDFC) _ integrated HBPNNs-HRBNs-SVM classifier. The experimental results show that the semi-supervised depth algorithm DBNESR has better, higher and stable recognition rate, and the semi-supervised learning algorithm has better effect than all of the supervised learning algorithms, and the integrated neural network has better effect than the single neural network. The average recognition rate and variance are BPHBPNNs, RBFHRBFNNs, SVMMCDFCDBNESR and BPRBHBPNNsHRBFNNsSVMMCDFCDBNESR, respectively; this reflects the ability of the DBNESR to simulate complex artificial intelligence tasks.
【學(xué)位授予單位】：華南理工大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2015
【分類號】：TP183
【共引文獻(xiàn)】
相關(guān)期刊論文 前6條
1 費(fèi)建超;芮挺;周怞;方虎生;朱會杰;;基于梯度的多輸入卷積神經(jīng)網(wǎng)絡(luò)[J];光電工程;2015年03期
2 薛皓天;楊晶東;談凱德;;一種改進(jìn)的BP神經(jīng)網(wǎng)絡(luò)在手寫體識別上的應(yīng)用[J];電子科技;2015年05期
3 劉建偉;崔立鵬;黎海恩;羅雄麟;;概率圖模型推理方法的研究進(jìn)展[J];計(jì)算機(jī)科學(xué);2015年04期
4 徐照松;元昌安;覃曉;元建;李雙;;基于關(guān)聯(lián)規(guī)則的語音情感中韻律特征抽取算法研究[J];計(jì)算機(jī)應(yīng)用與軟件;2015年09期
5 張雪英;孫穎;張衛(wèi);暢江;;語音情感識別的關(guān)鍵技術(shù)[J];太原理工大學(xué)學(xué)報(bào);2015年06期
6 康巖;盧慕超;閻高偉;;基于DBN-ELM的球磨機(jī)料位軟測量方法研究[J];儀表技術(shù)與傳感器;2015年04期
相關(guān)會議論文 前1條
1 黎海恩;劉建偉;羅雄麟;;概率圖模型的變分近似推理[A];2013年中國智能自動化學(xué)術(shù)會議論文集（第四分冊）[C];2013年
相關(guān)碩士學(xué)位論文 前7條
1 周曉鳳;家庭服務(wù)機(jī)器人的語音情感識別方法研究[D];華南理工大學(xué);2013年
2 佘洪新;基于臉型的三維人臉識別方法研究[D];華中師范大學(xué);2014年
3 程文華;基于機(jī)器學(xué)習(xí)的手機(jī)游戲軟件自動化測試研究[D];浙江大學(xué);2014年
4 沈燕;采用GSWM參數(shù)的多分類器融合語音情感識別[D];蘇州大學(xué);2014年
5 朱啟敏;基于云計(jì)算平臺的神經(jīng)網(wǎng)絡(luò)計(jì)算方法及其應(yīng)用研究[D];華南理工大學(xué);2014年
6 康巖;深度學(xué)習(xí)在球磨機(jī)料位軟測量建模中的應(yīng)用研究[D];太原理工大學(xué);2014年
7 徐照松;關(guān)聯(lián)規(guī)則挖掘在語音情感識別中的應(yīng)用研究[D];廣西師范學(xué)院;2014年
，本文編號：2482489
本文鏈接：http://sikaile.net/shoufeilunwen/xxkjbs/2482489.html
最近更新
教材專著
