本文關(guān)鍵詞：基于交叉熵的隨機(jī)賦權(quán)網(wǎng)絡(luò)　出處：《河北大學(xué)》2017年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 極速學(xué)習(xí)機(jī) 過擬合 均方誤差損失函數(shù) 交叉熵

【摘要】：近年來,隨著信息技術(shù)與計(jì)算機(jī)應(yīng)用技術(shù)的不斷進(jìn)步發(fā)展,整個(gè)社會(huì)進(jìn)入了大數(shù)據(jù)時(shí)代.因此,如何利用當(dāng)前先進(jìn)的數(shù)據(jù)分析技術(shù),從海量的數(shù)據(jù)中挖掘出所需的信息成為最關(guān)鍵的問題.分類問題作為數(shù)據(jù)分析中的主要問題也在不斷地引起人們的關(guān)注.黃廣斌提出了一種結(jié)構(gòu)簡單的神經(jīng)網(wǎng)絡(luò):極速學(xué)習(xí)機(jī),它是基于最小均方誤差的原則求得矩陣廣義逆,具有訓(xùn)練時(shí)間短,測試精度較高的優(yōu)點(diǎn).但是,因?yàn)镋LM僅考慮的是訓(xùn)練數(shù)據(jù)經(jīng)驗(yàn)誤差最小化,容易產(chǎn)生過擬合現(xiàn)象.本文的主要工作是:提出用交叉熵?fù)p失函數(shù)替代均方誤差損失函數(shù),在神經(jīng)網(wǎng)絡(luò)產(chǎn)生過擬合情況下,比較兩者的測試精度,以此來比較二者的泛化能力.具體地,過擬合現(xiàn)象是機(jī)器學(xué)習(xí)中一種常見的現(xiàn)象,表現(xiàn)為分類器能夠100%的正確分類訓(xùn)練樣本數(shù)據(jù),但對(duì)于其他數(shù)據(jù)則表現(xiàn)較差,其原因是構(gòu)造的函數(shù)過于精細(xì)復(fù)雜.在ELM中,通過計(jì)算隱藏輸出矩陣的廣義逆,找到具有最小二范數(shù)的最優(yōu)解.但由于隱層輸出矩陣的行數(shù)遠(yuǎn)遠(yuǎn)大于列數(shù),即隱層節(jié)點(diǎn)的數(shù)量很多,會(huì)出現(xiàn)過擬合現(xiàn)象.為了解決這個(gè)問題,本文提出一種基于交叉熵的隨機(jī)賦權(quán)網(wǎng)絡(luò)(CE-RWNNs),用交叉熵最小化原理代替均方誤差最小化原理.實(shí)驗(yàn)結(jié)果證明,提出的CE-RWNNs可以一定程度上克服在具有許多隱層節(jié)點(diǎn)的ELM中過擬合的缺點(diǎn).
[Abstract]:In recent years, with the continuous development of information technology and computer application technology, the whole society has entered the big data era. Therefore, how to use the current advanced data analysis technology. Mining the needed information from massive data has become the most critical issue. Classification, as the main problem in data analysis, has been attracting more and more attention. Huang Guangbin proposed a simple neural network. :. Speed learning machine. It is based on the principle of minimum mean square error to obtain matrix generalized inverse, which has the advantages of short training time and high test accuracy. However, because ELM only considers the minimum empirical error of training data. The main work of this paper is to use the cross-entropy loss function to replace the mean square error loss function. Specifically, over-fitting is a common phenomenon in machine learning, which shows that the classifier can correctly classify the training sample data of 100%. But for other data, the reason is that the constructed function is too fine and complex. In ELM, the generalized inverse of hidden output matrix is calculated. Find the optimal solution with least square norm, but because the number of rows of hidden layer output matrix is far larger than the number of columns, that is, there are many hidden layer nodes, there will be a phenomenon of over-fitting. In order to solve this problem. In this paper, a random weight network based on cross-entropy is proposed. The principle of cross-entropy minimization is used to replace the principle of mean square error minimization. The proposed CE-RWNNs can overcome the shortcoming of overfitting in ELM with many hidden nodes to some extent.
【學(xué)位授予單位】：河北大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP18

【參考文獻(xiàn)】

相關(guān)期刊論文 前5條

1 張沛洲;王熙照;顧迪;趙士欣;;基于共軛梯度的極速學(xué)習(xí)機(jī)[J];計(jì)算機(jī)應(yīng)用;2015年10期

2 吳木貴;江彩英;張信華;賴榮欽;;交叉熵神經(jīng)網(wǎng)絡(luò)及其在閩北大雨以上降水預(yù)報(bào)中的應(yīng)用[J];南京信息工程大學(xué)學(xué)報(bào)(自然科學(xué)版);2012年03期

3 全宇;王忠慶;何苗;;基于交叉熵的神經(jīng)網(wǎng)絡(luò)在病理圖像分析中的應(yīng)用[J];中國醫(yī)科大學(xué)學(xué)報(bào);2009年06期

4 劉R，

本文編號(hào)：1391759