本文選題：文本分類 + SVM　； 參考：《吉林大學(xué)》2017年碩士論文

【摘要】：隨著社會科技的進步,各個領(lǐng)域?qū)?shù)據(jù)的關(guān)注度與日俱增,與此同時,科研人員對數(shù)據(jù)的敏感性和對數(shù)據(jù)的應(yīng)用能力也不斷增強,這一切使人們進入了大數(shù)據(jù)時代。但是在互聯(lián)網(wǎng)中流動的不只有人們需要的可用資源,還包括大量干擾正常工作、誤導(dǎo)大眾的危害內(nèi)容。在可用資源里,數(shù)據(jù)也是雜亂無章的,這不僅造成網(wǎng)絡(luò)信息過載,也給人們帶來了低效率的感受。因此,對數(shù)據(jù)進行系統(tǒng)的處理、精準的分類,使它們成為有特定用途的可用信息是科研人員的追求目標。本文在撰寫的前期,先就當前文本分類的研究成果進行了一定程度的學(xué)習(xí),這其中包括對國內(nèi)和國外兩部分成果的研究;然后,著重學(xué)習(xí)和分析了如何用SVM方法解決文本二分類問題,進而引申到多分類問題。SVM——支持向量機,屬于機器學(xué)習(xí)中的一種方法,是以統(tǒng)計學(xué)習(xí)理論作為基礎(chǔ)的,在文本分類、圖像分類等許多領(lǐng)域都體現(xiàn)了很好的性能。在使用分類器之前,需要準備可靠的數(shù)據(jù)作為輸入,以保證分類的高效性。本文通過學(xué)習(xí)與分析,決定在文本表示階段做出一定的改變。文本在成為計算機能夠識別的形式時,需要對自身的表現(xiàn)形式做某種轉(zhuǎn)化。轉(zhuǎn)化的方式有很多,可以把詞轉(zhuǎn)化成向量,或者最簡單的二進制格式等。綜合詞語的語義和出現(xiàn)頻率兩方面因素,本文決定使用doc2vec算法作為文本表示方法。為此,本文的整體撰寫框架如下:首先,對文本分類問題的研究現(xiàn)狀和整體發(fā)展過程進行學(xué)習(xí)后,對本文的實驗?zāi)康暮拖敕ㄗ隽巳娴姆治?明確了理論框架和實驗流程。主要包括:對信息進行預(yù)處理,其分為文本的特征表示和特征提取兩部分;接著對幾種經(jīng)典的分類器算法進行介紹,著重分析了支持向量機的基本原理。然后,介紹深度學(xué)習(xí)的主要內(nèi)容和word2vec算法,以及在此算法基礎(chǔ)上發(fā)展而來的doc2vec算法,對詞向量模型進行比較,確定實驗所使用的模型。最后,將實驗需要的理論基礎(chǔ)和思想介紹完畢后,將理論與實踐結(jié)合,設(shè)計一個基于SVM的中文新聞文本分類模型。該模型的主要內(nèi)容是:以doc2vec的輸出作為多核SVM的輸入,利用實驗語料集,計算多個和矩陣,最后使用spg-gmkl訓(xùn)練并分類,實驗結(jié)果可以證明多核SVM的優(yōu)勢與實用性。
[Abstract]:With the development of social science and technology, more and more attention has been paid to the data in every field. Meanwhile, the sensitivity of the researchers to the data and their ability to apply the data have been enhanced, which has made people enter the era of big data. But what flows through the Internet is not only the available resources that people need, but also a lot of harmful content that interferes with normal work and misleads the public. In the available resources, the data is also messy, which not only causes the network information overload, but also brings people the feeling of inefficiency. Therefore, systematic processing and accurate classification of data is the goal of scientific researchers. In the early stage of writing, this paper first studies the current research results of text classification to a certain extent, which includes the domestic and foreign two parts of the research; then, This paper focuses on the study and analysis of how to solve the second classification problem of text by using SVM method, and then extends to the multi-classification problem. SVM-support vector machine (SVM), which is a method in machine learning, is based on the statistical learning theory, and is based on the text classification. Many fields, such as image classification, show good performance. Before using classifier, we need to prepare reliable data as input to ensure the efficiency of classification. Through study and analysis, this paper decides to make some changes in the text representation stage. When the text becomes a form that the computer can recognize, it needs to make some transformation to its own form of expression. There are many ways to convert words into vectors, or the simplest binary format. Considering the semantic and frequency of words, this paper decides to use doc2vec algorithm as a text representation method. Therefore, the overall writing framework of this paper is as follows: first, after studying the current situation and the overall development process of text classification, this paper makes a comprehensive analysis of the purpose and ideas of the experiment, and clarifies the theoretical framework and experimental flow. The main contents are as follows: preprocessing of information, which is divided into two parts: feature representation and feature extraction. Then, several classical classifier algorithms are introduced, and the basic principle of support vector machine is analyzed emphatically. Then, this paper introduces the main contents of in-depth learning and word2vec algorithm, and the doc2vec algorithm developed on the basis of this algorithm, compares the word vector model and determines the model used in the experiment. Finally, after introducing the theoretical basis and ideas needed by the experiment, a Chinese news text classification model based on SVM is designed by combining theory with practice. The main contents of this model are as follows: the output of doc2vec is taken as the input of multi-core SVM, and several sum matrices are calculated by using experimental corpus. Finally, the advantages and practicability of multi-core SVM are proved by using spg-gmkl training and classification.
【學(xué)位授予單位】：吉林大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP391.1

【參考文獻】

相關(guān)期刊論文 前10條

1 唐明;朱磊;鄒顯春;;基于Word2Vec的一種文檔向量表示[J];計算機科學(xué);2016年06期

2 DENG ChenWei;HUANG GuangBin;XU Jia;TANG JieXiong;;Extreme learning machines: new trends and applications[J];Science China(Information Sciences);2015年02期

3 谷軍;何南;;基于特征詞權(quán)值的漁業(yè)文本分類研究[J];大眾科技;2014年12期

4 張浩;吳秀娟;;深度學(xué)習(xí)的內(nèi)涵及認知理論基礎(chǔ)探析[J];中國電化教育;2012年10期

5 胡瀚;;基于MKL-SVM的網(wǎng)絡(luò)購物評論分類方法[J];計算機時代;2012年04期

6 華秀麗;朱巧明;李培峰;;語義分析與詞頻統(tǒng)計相結(jié)合的中文文本相似度量方法研究[J];計算機應(yīng)用研究;2012年03期

7 林秋蝦;;基于支持向量機的中文分詞[J];現(xiàn)代計算機(專業(yè)版);2011年23期

8 武征鵬;張學(xué)工;;Feature Rescaling of Support Vector Machines[J];Tsinghua Science and Technology;2011年04期

9 蘇喻;鄭誠;馬中杰;;基于語義的VSM模型改進[J];計算機應(yīng)用與軟件;2011年08期

10 黃承慧;印鑒;侯f ;;一種結(jié)合詞項語義信息和TF-IDF方法的文本相似度量方法[J];計算機學(xué)報;2011年05期

相關(guān)碩士學(xué)位論文 前3條

1 張志華;基于深度學(xué)習(xí)的情感詞向量及文本情感分析的研究[D];華東師范大學(xué);2016年

2 崔清亮;多核學(xué)習(xí)方法在分類問題中的應(yīng)用研究[D];蘭州交通大學(xué);2014年

3 卜凡軍;KNN算法的改進及其在文本分類中的應(yīng)用[D];江南大學(xué);2009年

，

本文編號：1925428