基于深度學(xué)習(xí)的領(lǐng)域適應(yīng)問題研究
發(fā)布時(shí)間：2018-08-19 06:33
【摘要】：在視覺領(lǐng)域中,收集充分標(biāo)注數(shù)據(jù)代價(jià)昂貴,而標(biāo)準(zhǔn)監(jiān)督學(xué)習(xí)在標(biāo)注數(shù)據(jù)稀缺時(shí)泛化能力差,無法滿足實(shí)際需求,領(lǐng)域適應(yīng)作為一個(gè)新興的機(jī)器學(xué)習(xí)方法,旨在利用有豐富標(biāo)簽的源領(lǐng)域數(shù)據(jù)訓(xùn)練分類器,用于無標(biāo)簽或少量標(biāo)簽的目標(biāo)領(lǐng)域。目前,跨領(lǐng)域遷移學(xué)習(xí)效果不理想的主要原因是存在負(fù)遷移、欠適配和欠擬合等三大問題,而另一種更具挑戰(zhàn)性的情況是源領(lǐng)域與目標(biāo)領(lǐng)域處于異構(gòu)特征空間,致使遷移更加困難。因此,針對(duì)上述問題,本文的主要研究?jī)?nèi)容為:第一,針對(duì)同構(gòu)領(lǐng)域適應(yīng),如何學(xué)習(xí)有效特征并最大程度減小領(lǐng)域間的分布差異以改善欠適配問題,提出了基于自動(dòng)編碼器的領(lǐng)域適應(yīng)網(wǎng)絡(luò)。首先,源域和目標(biāo)域樣本分別經(jīng)過兩層編碼和解碼操作以最小化重構(gòu)誤差學(xué)習(xí)更有效的特征表達(dá);然后,分別在特征提取層和分類層使用最大均值差異準(zhǔn)則同時(shí)匹配領(lǐng)域間的邊緣和條件分布以最小化分布差異,并使用softmax分類器將源數(shù)據(jù)標(biāo)簽信息編碼以提高分類表現(xiàn);最后,通過梯度下降法學(xué)習(xí)網(wǎng)絡(luò)參數(shù),根據(jù)分類器的輸出完成對(duì)目標(biāo)域無標(biāo)簽樣本的預(yù)測(cè)。第二,針對(duì)同構(gòu)領(lǐng)域適應(yīng),學(xué)習(xí)模型未能充分描述預(yù)測(cè)數(shù)據(jù)所服從的概率分布而同時(shí)導(dǎo)致欠擬合和欠適配問題,且普通圖正則項(xiàng)的引入未能充分改善負(fù)遷移問題,提出了基于超圖正則化降噪自動(dòng)編碼器的領(lǐng)域適應(yīng)網(wǎng)絡(luò)。首先,通過降噪自動(dòng)編碼器提取更具魯棒性的特征以減小欠擬合問題;其次,使用最大均值差異準(zhǔn)則同時(shí)匹配領(lǐng)域間的邊緣和條件分布以解決欠適配問題;然后,根據(jù)源和目標(biāo)領(lǐng)域樣本間關(guān)系引入超圖正則項(xiàng)以解決負(fù)遷移問題,并根據(jù)源領(lǐng)域真實(shí)標(biāo)簽得到分類器損失函數(shù);最后,通過梯度下降法學(xué)習(xí)網(wǎng)絡(luò)參數(shù),完成目標(biāo)域樣本的分類。第三,針對(duì)異構(gòu)領(lǐng)域適應(yīng),淺層結(jié)構(gòu)無法很好地?cái)M合數(shù)據(jù)分布并得到更有效的特征表達(dá),且未同時(shí)考慮到領(lǐng)域間數(shù)據(jù)分布的匹配以及幾何結(jié)構(gòu)和標(biāo)簽的一致性,提出基于自動(dòng)編碼器的異構(gòu)領(lǐng)域適應(yīng)網(wǎng)絡(luò)。首先,分別利用兩組自動(dòng)編碼器將源和目標(biāo)領(lǐng)域數(shù)據(jù)映射到共享特征空間,并使用最大均值差異準(zhǔn)則同時(shí)匹配領(lǐng)域間的邊緣與條件分布;其次,引入流形對(duì)齊項(xiàng),其中幾何項(xiàng)用以保持領(lǐng)域內(nèi)數(shù)據(jù)幾何結(jié)構(gòu)的一致性,相似項(xiàng)和相異項(xiàng)則用以保持領(lǐng)域間標(biāo)簽信息的一致性;然后,利用源領(lǐng)域和目標(biāo)領(lǐng)域的標(biāo)簽信息,得到softmax分類器損失項(xiàng);最后,通過梯度下降法學(xué)習(xí)網(wǎng)絡(luò)參數(shù),實(shí)現(xiàn)對(duì)目標(biāo)域無標(biāo)簽樣本的分類。在多個(gè)數(shù)據(jù)集上進(jìn)行的對(duì)比實(shí)驗(yàn)結(jié)果表明,與傳統(tǒng)同構(gòu)和異構(gòu)領(lǐng)域適應(yīng)方法相比,本文所提模型均能夠獲得更好的分類表現(xiàn),有效解決跨領(lǐng)域知識(shí)遷移問題。
[Abstract]:In the field of vision, it is expensive to collect fully annotated data, while standard supervised learning has poor generalization ability when tagging data is scarce, so it can not meet the actual needs. Therefore, domain adaptation is a new machine learning method. The aim of this paper is to use source domain data with rich tags to train classifiers for target areas without or with a few tags. At present, the main reasons why the effect of cross-domain transfer learning is not ideal are that there are three major problems: negative transfer, inadequate adaptation and under-fitting. Another more challenging situation is that the source domain and the target domain are in heterogeneous feature space. This makes migration more difficult. Therefore, the main research contents of this paper are as follows: first, how to learn effective features and minimize the distribution differences between domains in order to improve the ill-fit for isomorphic domain adaptation. A domain adaptive network based on automatic encoder is proposed. First, the source domain and target domain samples are encoded and decoded by two layers respectively to minimize the reconstruction error to learn more efficient feature representation. In the feature extraction layer and classification layer, the maximum mean difference criterion is used to match the edge and conditional distribution of the domain simultaneously to minimize the distribution differences, and the source data label information is encoded by the softmax classifier to improve the classification performance. The network parameters are studied by gradient descent method, and the target domain unlabeled samples are predicted according to the output of the classifier. Secondly, for the adaptation of isomorphism domain, the learning model can not adequately describe the probability distribution of the predicted data, which leads to the problem of under-fitting and ill-fit, and the introduction of the regular term in the common graph can not fully improve the negative migration problem. A domain adaptive network based on hypergraph regularization noise reduction automatic encoder is proposed. Firstly, the noise reduction automatic encoder is used to extract the more robust features to reduce the underfitting problem. Secondly, the maximum mean difference criterion is used to simultaneously match the edge and conditional distribution between domains to solve the problem of inadequate matching. According to the relationship between source and target domain samples, hypergraph canonical items are introduced to solve the problem of negative migration, and the classifier loss function is obtained according to the real label of source domain. Finally, the classification of target domain samples is accomplished by learning network parameters by gradient descent method. Thirdly, for heterogeneous domain adaptation, shallow structure can not fit the data distribution well and get more effective feature representation, and does not take into account the matching of data distribution between domains and the consistency of geometric structure and label at the same time. A heterogeneous domain adaptive network based on automatic encoder is proposed. Firstly, two sets of automatic encoders are used to map the source and target domain data to the shared feature space, and the maximum mean difference criterion is used to match the edge and conditional distribution of the domain simultaneously. The geometric terms are used to maintain the consistency of the geometric structure of the data in the domain, the similarity items and the different items are used to maintain the consistency of the label information between the domains, and then the loss items of the softmax classifier are obtained by using the label information of the source domain and the target domain. Finally, the network parameters are studied by gradient descent method to realize the classification of untagged samples in target domain. The experimental results on multiple datasets show that compared with the traditional isomorphism and heterogeneous domain adaptation methods, the proposed models can achieve better classification performance and effectively solve the problem of cross-domain knowledge transfer.
【學(xué)位授予單位】：中國(guó)礦業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP391.41;TP181
本文編號(hào)：2190946
[Abstract]:In the field of vision, it is expensive to collect fully annotated data, while standard supervised learning has poor generalization ability when tagging data is scarce, so it can not meet the actual needs. Therefore, domain adaptation is a new machine learning method. The aim of this paper is to use source domain data with rich tags to train classifiers for target areas without or with a few tags. At present, the main reasons why the effect of cross-domain transfer learning is not ideal are that there are three major problems: negative transfer, inadequate adaptation and under-fitting. Another more challenging situation is that the source domain and the target domain are in heterogeneous feature space. This makes migration more difficult. Therefore, the main research contents of this paper are as follows: first, how to learn effective features and minimize the distribution differences between domains in order to improve the ill-fit for isomorphic domain adaptation. A domain adaptive network based on automatic encoder is proposed. First, the source domain and target domain samples are encoded and decoded by two layers respectively to minimize the reconstruction error to learn more efficient feature representation. In the feature extraction layer and classification layer, the maximum mean difference criterion is used to match the edge and conditional distribution of the domain simultaneously to minimize the distribution differences, and the source data label information is encoded by the softmax classifier to improve the classification performance. The network parameters are studied by gradient descent method, and the target domain unlabeled samples are predicted according to the output of the classifier. Secondly, for the adaptation of isomorphism domain, the learning model can not adequately describe the probability distribution of the predicted data, which leads to the problem of under-fitting and ill-fit, and the introduction of the regular term in the common graph can not fully improve the negative migration problem. A domain adaptive network based on hypergraph regularization noise reduction automatic encoder is proposed. Firstly, the noise reduction automatic encoder is used to extract the more robust features to reduce the underfitting problem. Secondly, the maximum mean difference criterion is used to simultaneously match the edge and conditional distribution between domains to solve the problem of inadequate matching. According to the relationship between source and target domain samples, hypergraph canonical items are introduced to solve the problem of negative migration, and the classifier loss function is obtained according to the real label of source domain. Finally, the classification of target domain samples is accomplished by learning network parameters by gradient descent method. Thirdly, for heterogeneous domain adaptation, shallow structure can not fit the data distribution well and get more effective feature representation, and does not take into account the matching of data distribution between domains and the consistency of geometric structure and label at the same time. A heterogeneous domain adaptive network based on automatic encoder is proposed. Firstly, two sets of automatic encoders are used to map the source and target domain data to the shared feature space, and the maximum mean difference criterion is used to match the edge and conditional distribution of the domain simultaneously. The geometric terms are used to maintain the consistency of the geometric structure of the data in the domain, the similarity items and the different items are used to maintain the consistency of the label information between the domains, and then the loss items of the softmax classifier are obtained by using the label information of the source domain and the target domain. Finally, the network parameters are studied by gradient descent method to realize the classification of untagged samples in target domain. The experimental results on multiple datasets show that compared with the traditional isomorphism and heterogeneous domain adaptation methods, the proposed models can achieve better classification performance and effectively solve the problem of cross-domain knowledge transfer.
【學(xué)位授予單位】：中國(guó)礦業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TP391.41;TP181
【參考文獻(xiàn)】
相關(guān)期刊論文 前6條
1 唐宋;陳利娟;陳志賢;葉茂;;基于目標(biāo)域局部近鄰幾何信息的域自適應(yīng)圖像分類方法[J];計(jì)算機(jī)應(yīng)用;2017年04期
2 奚雪峰;周國(guó)棟;;面向自然語言處理的深度學(xué)習(xí)研究[J];自動(dòng)化學(xué)報(bào);2016年10期
3 臧紹飛;程玉虎;王雪松;;基于最大分布加權(quán)均值嵌入的領(lǐng)域適應(yīng)學(xué)習(xí)[J];控制與決策;2016年11期
4 周軍娜;陳偉;王珂;湯鎮(zhèn)宇;;通過域適應(yīng)實(shí)現(xiàn)人臉識(shí)別[J];計(jì)算機(jī)應(yīng)用研究;2017年06期
5 孟娟;胡谷雨;潘志松;周宇歡;;領(lǐng)域適應(yīng)學(xué)習(xí)算法研究與展望[J];計(jì)算機(jī)科學(xué);2015年10期
6 鞏萍;程玉虎;王雪松;;基于屬性關(guān)系圖正則化特征選擇的零樣本分類[J];中國(guó)礦業(yè)大學(xué)學(xué)報(bào);2015年06期
相關(guān)碩士學(xué)位論文 前1條
1 毛海斌;基于半監(jiān)督機(jī)器學(xué)習(xí)的情感分類領(lǐng)域適應(yīng)問題研究[D];南京理工大學(xué);2015年
，本文編號(hào)：2190946
本文鏈接：http://sikaile.net/kejilunwen/zidonghuakongzhilunwen/2190946.html
最近更新
教材專著
