本文關鍵詞：基于交叉熵的隨機賦權網(wǎng)絡　出處：《河北大學》2017年碩士論文　論文類型：學位論文

  更多相關文章： 極速學習機 過擬合 均方誤差損失函數(shù) 交叉熵

【摘要】：近年來,隨著信息技術與計算機應用技術的不斷進步發(fā)展,整個社會進入了大數(shù)據(jù)時代.因此,如何利用當前先進的數(shù)據(jù)分析技術,從海量的數(shù)據(jù)中挖掘出所需的信息成為最關鍵的問題.分類問題作為數(shù)據(jù)分析中的主要問題也在不斷地引起人們的關注.黃廣斌提出了一種結構簡單的神經(jīng)網(wǎng)絡:極速學習機,它是基于最小均方誤差的原則求得矩陣廣義逆,具有訓練時間短,測試精度較高的優(yōu)點.但是,因為ELM僅考慮的是訓練數(shù)據(jù)經(jīng)驗誤差最小化,容易產(chǎn)生過擬合現(xiàn)象.本文的主要工作是:提出用交叉熵損失函數(shù)替代均方誤差損失函數(shù),在神經(jīng)網(wǎng)絡產(chǎn)生過擬合情況下,比較兩者的測試精度,以此來比較二者的泛化能力.具體地,過擬合現(xiàn)象是機器學習中一種常見的現(xiàn)象,表現(xiàn)為分類器能夠100%的正確分類訓練樣本數(shù)據(jù),但對于其他數(shù)據(jù)則表現(xiàn)較差,其原因是構造的函數(shù)過于精細復雜.在ELM中,通過計算隱藏輸出矩陣的廣義逆,找到具有最小二范數(shù)的最優(yōu)解.但由于隱層輸出矩陣的行數(shù)遠遠大于列數(shù),即隱層節(jié)點的數(shù)量很多,會出現(xiàn)過擬合現(xiàn)象.為了解決這個問題,本文提出一種基于交叉熵的隨機賦權網(wǎng)絡(CE-RWNNs),用交叉熵最小化原理代替均方誤差最小化原理.實驗結果證明,提出的CE-RWNNs可以一定程度上克服在具有許多隱層節(jié)點的ELM中過擬合的缺點.
[Abstract]:In recent years, with the continuous development of information technology and computer application technology, the whole society has entered the big data era. Therefore, how to use the current advanced data analysis technology. Mining the needed information from massive data has become the most critical issue. Classification, as the main problem in data analysis, has been attracting more and more attention. Huang Guangbin proposed a simple neural network. :. Speed learning machine. It is based on the principle of minimum mean square error to obtain matrix generalized inverse, which has the advantages of short training time and high test accuracy. However, because ELM only considers the minimum empirical error of training data. The main work of this paper is to use the cross-entropy loss function to replace the mean square error loss function. Specifically, over-fitting is a common phenomenon in machine learning, which shows that the classifier can correctly classify the training sample data of 100%. But for other data, the reason is that the constructed function is too fine and complex. In ELM, the generalized inverse of hidden output matrix is calculated. Find the optimal solution with least square norm, but because the number of rows of hidden layer output matrix is far larger than the number of columns, that is, there are many hidden layer nodes, there will be a phenomenon of over-fitting. In order to solve this problem. In this paper, a random weight network based on cross-entropy is proposed. The principle of cross-entropy minimization is used to replace the principle of mean square error minimization. The proposed CE-RWNNs can overcome the shortcoming of overfitting in ELM with many hidden nodes to some extent.
【學位授予單位】：河北大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TP18

【參考文獻】

相關期刊論文 前5條

1 張沛洲;王熙照;顧迪;趙士欣;;基于共軛梯度的極速學習機[J];計算機應用;2015年10期

2 吳木貴;江彩英;張信華;賴榮欽;;交叉熵神經(jīng)網(wǎng)絡及其在閩北大雨以上降水預報中的應用[J];南京信息工程大學學報(自然科學版);2012年03期

3 全宇;王忠慶;何苗;;基于交叉熵的神經(jīng)網(wǎng)絡在病理圖像分析中的應用[J];中國醫(yī)科大學學報;2009年06期

4 劉R，

本文編號：1391759