【摘要】：支持向量機(jī)是近幾十年來(lái)在機(jī)器學(xué)習(xí)方向最重要的進(jìn)展之一。它從一開始出現(xiàn)就因其十分優(yōu)秀的分類能力得到了眾多國(guó)內(nèi)外研究學(xué)者的關(guān)注。現(xiàn)今為止已經(jīng)被應(yīng)用到許許多多的領(lǐng)域發(fā)揮著巨大的作用。因此各類支持向量機(jī)的求解算法也成為了廣大學(xué)者的研究重點(diǎn)。特別是順序最小優(yōu)化算法更是受到廣泛的關(guān)注。順序最小優(yōu)化算法具有優(yōu)美的二次規(guī)劃表達(dá),從而避免了對(duì)空間過(guò)大的需求,使實(shí)現(xiàn)支持向量機(jī)的過(guò)程變得簡(jiǎn)單而高效。但是即便如此,順序最小優(yōu)化算法本身的效率提升問(wèn)題也是現(xiàn)今研究的重點(diǎn)。本文在傳統(tǒng)的順序最小優(yōu)化算法的基礎(chǔ)上,分別通過(guò)大量實(shí)驗(yàn)借助目標(biāo)函數(shù)值和間隔值的變化對(duì)支持向量機(jī)以及順序最小優(yōu)化算法的求解過(guò)程進(jìn)行了深入地分析,并且根據(jù)分析結(jié)果對(duì)算法求解過(guò)程中的停止條件進(jìn)行了改進(jìn)。改進(jìn)過(guò)程中對(duì)目標(biāo)函數(shù)值和間隔值變化曲線進(jìn)行了平滑處理,統(tǒng)計(jì)數(shù)據(jù)來(lái)對(duì)兩種改進(jìn)后的順序最小優(yōu)化算法的效果進(jìn)行衡量,且進(jìn)一步采用交叉驗(yàn)證的方法驗(yàn)證改進(jìn)算法的結(jié)果。本文的研究工作主要包括:(1)依據(jù)傳統(tǒng)的順序最小優(yōu)化算法推導(dǎo)出目標(biāo)函數(shù)值及間隔值的表達(dá)式,并編寫相應(yīng)代碼,能夠分別輸出目標(biāo)函數(shù)值以及間隔值隨迭代次數(shù)變化的數(shù)據(jù)。對(duì)代碼改進(jìn)后的傳統(tǒng)的順序最小優(yōu)化算法過(guò)程進(jìn)行大量實(shí)驗(yàn)研究,借助目標(biāo)函數(shù)值和間隔值觀察過(guò)程中的每一個(gè)量的變化情況。(2)實(shí)驗(yàn)過(guò)程中對(duì)目標(biāo)函數(shù)值和間隔值的變化曲線分別進(jìn)行平滑處理,并用更具形象曲線變化的方式表現(xiàn)其變化過(guò)程。發(fā)現(xiàn)目標(biāo)函數(shù)值和間隔值隨迭代次數(shù)變化的規(guī)律:目標(biāo)函數(shù)值及間隔值隨迭代次數(shù)的變化類似均呈鉸鏈函數(shù)形態(tài),有一個(gè)明顯的拐點(diǎn),在一定的迭代次數(shù)后(即拐點(diǎn)后)目標(biāo)函數(shù)值在很長(zhǎng)的一段時(shí)間里變化甚微,在目標(biāo)函數(shù)值的變化過(guò)程中甚至出現(xiàn)微小的升降波動(dòng)現(xiàn)象。(3)對(duì)傳統(tǒng)的順序最小優(yōu)化算法進(jìn)行過(guò)程改進(jìn)。分別進(jìn)行了目標(biāo)函數(shù)值輔助的順序最小優(yōu)化算法改進(jìn)以及間隔值輔助的順序最小優(yōu)化算法改進(jìn)。根據(jù)目標(biāo)函數(shù)值以及間隔值的曲線形態(tài),找到一個(gè)可以提前終止訓(xùn)練避免后期無(wú)效率訓(xùn)練同時(shí)又對(duì)訓(xùn)練的正確率影響不大的停止標(biāo)準(zhǔn),完成改進(jìn)代碼的編寫。(4)分別對(duì)兩個(gè)改進(jìn)后的順序最小優(yōu)化算法進(jìn)行實(shí)驗(yàn),統(tǒng)計(jì)數(shù)據(jù),并從訓(xùn)練效率以及測(cè)試的正確率方面對(duì)改進(jìn)算法和傳統(tǒng)算法的實(shí)驗(yàn)結(jié)果進(jìn)行比較將。同時(shí)采用比較權(quán)威的交叉驗(yàn)證的方法進(jìn)行訓(xùn)練效率和模型預(yù)測(cè)能力的進(jìn)一步比較,綜合分析改進(jìn)后算法的訓(xùn)練效果。通過(guò)大量的實(shí)驗(yàn)分析驗(yàn)證,本文所采取的新改進(jìn)的間隔值和目標(biāo)函數(shù)值輔助的順序最小優(yōu)化改進(jìn)算法在訓(xùn)練效率和模型預(yù)測(cè)能力比教優(yōu)越,同時(shí)本文兩種改進(jìn)算法比較來(lái)看,間隔值輔助的順序最小優(yōu)化算法對(duì)訓(xùn)練效率的提升更顯著。
[Abstract]:Support vector machine (SVM) is one of the most important advances in machine learning in recent decades. From the beginning, it has attracted the attention of many scholars at home and abroad because of its excellent classification ability. Up to now, it has been applied to many fields to play a great role. Therefore, all kinds of support vector machine algorithms have also become the focus of the majority of scholars. In particular, the sequential minimum optimization algorithm has attracted more and more attention. The sequential minimum optimization algorithm has a beautiful quadratic programming representation, which avoids the need for too much space and makes the process of implementing support vector machine simple and efficient. But even so, the efficiency improvement of sequential minimum optimization algorithm itself is the focus of current research. In this paper, on the basis of traditional sequential minimum optimization algorithm, the solving process of support vector machine and sequential minimum optimization algorithm are deeply analyzed by means of a large number of experiments with the help of the change of objective function value and interval value, respectively. According to the analysis results, the stopping condition of the algorithm is improved. In the process of improvement, the change curves of objective function value and interval value are smoothed, and the statistical data are used to measure the effect of the two improved sequential minimum optimization algorithms. Furthermore, cross-validation is used to verify the results of the improved algorithm. The main work of this paper is as follows: (1) according to the traditional sequential minimum optimization algorithm, the expressions of objective function value and interval value are deduced, and the corresponding code is written. Can output the object function values and interval values with the number of iterations data. A large number of experiments have been carried out on the traditional sequential minimum optimization algorithm process after code improvement. With the help of objective function value and interval value, the variation of each quantity in the process is observed. (2) the variation curves of objective function value and interval value are smoothed respectively during the experiment. And use more vivid curve change way to show its change process. It is found that the value of objective function and interval value vary with the number of iterations: the change of value of objective function and interval value with the number of iterations is similar to that of hinge function, and there is an obvious inflection point. After a certain number of iterations (that is, after the inflection point), the value of the objective function varies little over a long period of time. There are even slight fluctuation phenomena in the course of the change of objective function value. (3) the traditional sequential minimum optimization algorithm is improved. The sequential minimum optimization algorithm assisted by objective function value and the sequential minimum optimization algorithm assisted by interval value are improved respectively. According to the curve form of objective function value and interval value, a stopping criterion is found, which can stop training early and avoid inefficiency training in later period, but has little influence on the correct rate of training. The improved code is written. (4) two improved sequential minimum optimization algorithms are tested and compared with the traditional algorithm in terms of the training efficiency and the test accuracy. (4) the experimental results of the improved algorithm are compared with those of the traditional algorithm in terms of the training efficiency and the accuracy of testing. (4) two improved sequential minimum optimization algorithms are tested and statistically analyzed. At the same time, a more authoritative cross-validation method is used to compare the training efficiency and the prediction ability of the model, and the training effect of the improved algorithm is comprehensively analyzed. Through a large number of experiments, it is proved that the improved sequential minimum optimization algorithm based on improved interval value and objective function value is superior to teaching in training efficiency and model prediction ability. At the same time, the comparison of the two improved algorithms in this paper shows that the improved algorithm is superior to the teaching method in training efficiency and model prediction ability. The sequential minimum optimization algorithm assisted by interval value can improve the training efficiency more significantly.
【學(xué)位授予單位】：山東師范大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP181

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 柴巖;王慶菊;;基于邊界向量的樣本取樣SMO算法[J];系統(tǒng)工程;2015年06期

2 趙長(zhǎng)春;姜曉愛(ài);金英漢;;非線性回歸支持向量機(jī)的SMO算法改進(jìn)[J];北京航空航天大學(xué)學(xué)報(bào);2014年01期

3 劉學(xué)藝;李平;郜傳厚;;極限學(xué)習(xí)機(jī)的快速留一交叉驗(yàn)證算法[J];上海交通大學(xué)學(xué)報(bào);2011年08期

4 顧亞祥;丁世飛;;支持向量機(jī)研究進(jìn)展[J];計(jì)算機(jī)科學(xué);2011年02期

5 濮定國(guó);金中;;新的拉格朗日乘子方法[J];同濟(jì)大學(xué)學(xué)報(bào)(自然科學(xué)版);2010年09期

6 歐陽(yáng)玉梅;方若森;馬志強(qiáng);;評(píng)估蛋白質(zhì)相互作用可信度的生物信息學(xué)方法[J];生命科學(xué);2008年03期

7 駱世廣;駱昌日;;加快SMO算法訓(xùn)練速度的策略研究[J];計(jì)算機(jī)工程與應(yīng)用;2007年33期

8 許建潮;張玉石;;回歸支持向量機(jī)SMO算法的改進(jìn)[J];計(jì)算機(jī)工程與應(yīng)用;2007年17期

9 孔銳,張冰;一種快速支持向量機(jī)增量學(xué)習(xí)算法[J];控制與決策;2005年10期

10 張召;黃國(guó)興;鮑鈺;;一種改進(jìn)的SMO算法[J];計(jì)算機(jī)科學(xué);2003年08期

相關(guān)博士學(xué)位論文 前1條

1 段會(huì)川;高斯核函數(shù)支持向量分類機(jī)超級(jí)參數(shù)有效范圍研究[D];山東師范大學(xué);2012年

，

本文編號(hào)：2444051