本文關(guān)鍵詞： 大規(guī)模 稀疏學(xué)習(xí) 支持向量機 最優(yōu)化 損失函數(shù) 數(shù)據(jù)挖掘 分類問題　出處：《北京交通大學(xué)》2017年博士論文　論文類型：學(xué)位論文

【摘要】：稀疏學(xué)習(xí)是一種有效處理冗余問題的方法。目前,稀疏優(yōu)化方法已廣泛應(yīng)用于信號壓縮感知、圖像處理等實際問題中,其理論和算法都在快速發(fā)展中。由于大規(guī)模數(shù)據(jù)挖掘問題往往具有冗余和稀疏的特點,因此稀疏優(yōu)化是處理大規(guī)模數(shù)據(jù)挖掘問題的上佳之選。而支持向量機作為通用的機器學(xué)習(xí)方法,具有堅實的統(tǒng)計學(xué)習(xí)理論基礎(chǔ),實際應(yīng)用效果好,使用方便,模型參數(shù)較少,在圖像、視頻、聲音、文本等不同領(lǐng)域得到了廣泛的應(yīng)用。國內(nèi)外關(guān)于大規(guī)模稀疏支持向量機的理論研究和方法并不成熟,缺乏理論基礎(chǔ)和模型算法,尚處于初始階段。比如:1)稀疏模型的有效性檢驗指標(biāo),即如何度量模型的稀疏程度以及稀疏效果的好壞問題等;2)大規(guī)模問題的稀疏模型缺乏統(tǒng)一的理論基礎(chǔ);3)大規(guī)模問題的稀疏優(yōu)化模型求解問題;4)拓展研究比較少,對其拓展有較大空間。我們擬從最優(yōu)化的角度對上述多方面進行系統(tǒng)研究。本文共分七章,組織結(jié)構(gòu)如下:第一章為引言部分,介紹本文的研究背景、研究意義、研究對象和主要工作概述。第二章詳細(xì)介紹與本文研究內(nèi)容密切相關(guān)的算法,包括標(biāo)準(zhǔn)的支持向量機(SVM)、最小二乘支持向量機(LSSVM)、基于Ramp損失函數(shù)的支持向量機(RSVM)、雙子支持向量機(TWSVM)、非平行支持向量機(NPSVM),并比較分析了他們的優(yōu)缺點。由于NPSVM具有更好的推廣能力,后面的研究內(nèi)容則重點圍繞NPSVM展開,一方面從理論上探索其統(tǒng)計學(xué)習(xí)理論基礎(chǔ),另一方面從方法上構(gòu)建更稀疏的、能處理大規(guī)模問題的NPSVM模型和算法。第三章針對分類問題,提出一個具有稀疏性和魯棒性的非平行超平面分類機—基于Ramp損失函數(shù)的非平行超平面SVM(RNPSVM)。RNPSVM在訓(xùn)練階段可以處理含有噪音和異常點的數(shù)據(jù),并含有較少的支持向量,從而增加了模型的稀疏程度,具有更好的推廣能力。針對該模型中非凸優(yōu)化問題的求解,我們引入了有效的CCCP策略。進一步,對該模型的稀疏性、復(fù)雜度、初始化等進行了理論分析,大量的數(shù)值實驗也驗證了該模型的有效性。第四章從U-SVM的角度構(gòu)建了NPSVM的結(jié)構(gòu)風(fēng)險最小化原則,給出了其相應(yīng)的統(tǒng)計學(xué)習(xí)理論解釋。之后從提升計算效率的角度出發(fā),分別給出了基于線性規(guī)劃形式的NPSVM和基于線性規(guī)劃形式的RNPSVM,為NPSVM方法處理更大規(guī)模的問題提供了可選擇的模型。第五章首先討論了 LSTWS VM和LSS VM的關(guān)系,證明LSS VM是LSTWS VM的退化情況。進一步,基于LSSVM,提出了一個新的稀疏和魯棒的最小二乘支持向量機RLSSVM。在原有稀疏模型ε-LSSVM基礎(chǔ)上,構(gòu)建并引入了一個新的基于ε-不敏感損失函數(shù)的Ramp損失函數(shù),新模型可以有效地對噪音抗干擾,并且具有更好的稀疏性。引入了CCCP策略來求解該模型中非凸優(yōu)化問題,不同數(shù)據(jù)集上的數(shù)值實驗證明了RLSSVM的有效性。第六章基于前面的NPS VM和RNPS VM,提出針對大規(guī)模線性分類問題的交替方向乘子法(ADMM),ADMM是目前處理大規(guī)模問題的有效優(yōu)化算法。通過將NPSVM和RNPS VM中的優(yōu)化問題構(gòu)造為ADMM可以求解的形式,實現(xiàn)了ADMM在這兩個算法上的應(yīng)用。大量的實驗證明了算法的有效性。最后一章總結(jié)了本文的主要工作以及取得的成果,并提出了進一步的研究方向。
[Abstract]:Sparse learning is an effective method for treatment of redundant problems. At present, the sparse optimization method has been widely used in signal compressed sensing, image processing and other practical problems, the theory and algorithm are in rapid development. Because of the characteristics of large-scale data mining problems is often redundant and sparse, and sparse optimization is the best choice to deal with large-scale the problem of data mining. And the support vector machine as a general machine learning method, has a solid statistical learning theory, the practical application effect is good, easy to use, model parameter, in the image, video, sound, text and other different fields. It has been widely applied at home and abroad on the theory and method of large scale sparse support vector machine is not mature, lack of theoretical basis and model algorithm, is still in the initial stage. For example: 1) test validity index sparse model, namely how to measure model The sparse degree and sparse effect quality problems; 2) sparse model of large-scale problems the lack of theoretical basis; 3) sparse optimization model for solving large-scale problems; 4) expand the research is relatively small, there is a large space for its development. We have to carry out systematic research on these aspects from the optimization point in this paper. The organizational structure is divided into seven chapters, as follows: the first chapter is the introduction part, introduces the research background, research significance, research object and main works are summarized. The second chapter introduces the research content closely related algorithms, including the standard support vector machine (SVM), least squares support vector machine (LSSVM), support vector Ramp based on loss function (RSVM), twin support vector machine (TWSVM), non parallel support vector machine (NPSVM), and compare their advantages and disadvantages are analyzed. Because NPSVM has better generalization ability, behind The research content mainly around the NPSVM, on the one hand to explore the theoretical basis of learning from the statistical theory, on the other hand, the construction method from more sparse, NPSVM model and algorithm can handle large-scale problems. The third chapter according to the classification problem, this paper proposes a sparse and robust non parallel hyperplanes classifier Ramp loss function based on non parallel hyperplanes SVM (RNPSVM) to deal with noise and outliers in the data can be in the training phase.RNPSVM, support vector and contains less, thus increasing the degree of sparsity model, has better generalization ability. For solving non convex optimization problem in the model, we introduce effective the strategy of CCCP. Further, the sparsity and complexity of the model, initialization is analyzed, a large number of numerical experiments have verified the validity of the model. In the fourth chapter, from the perspective of U-SVM. The structural risk minimization principle NPSVM, explain the statistical learning theory. From the efficiency point of view, are given based on NPSVM linear programming form and linear programming based on the form of RNPSVM, the choice of model for the NPSVM method to handle larger scale problems. The fifth chapter discussed the relationship between LSTWS VM and LSS VM, LSS VM LSTWS that is a degenerate case of VM. Further, based on LSSVM, proposes a new sparse least squares and robust support vector machine RLSSVM. in the original model based on sparse epsilon -LSSVM, constructed and introduced a Ramp loss function insensitive loss function based on the new model can effectively resist interference to noise, and has better sparsity. Using CCCP method to solve the model of non convex optimization problem on the set of different numerical data The experiment proves the validity of the RLSSVM. The sixth chapter is based on the front of the NPS VM and RNPS VM, is proposed to solve the classification problem of large-scale linear alternating direction method of multipliers (ADMM, ADMM) is an effective optimization algorithm currently dealing with large-scale problems. Through the optimization of NPSVM and RNPS in VM structure for ADMM can be solved in the form of and realizes the application of ADMM in these two algorithms. Experiments prove the effectiveness of the algorithm. The last chapter summarizes the main work and achievements, and puts forward the direction of further research.

【學(xué)位授予單位】：北京交通大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2017
【分類號】：TP18

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 TIAN YingJie;JU XuChan;QI ZhiQuan;SHI Yong;;Improved twin support vector machine[J];Science China(Mathematics);2014年02期

，

本文編號：1441992