本文選題：Spark分布式內(nèi)存計(jì)算框架　切入點(diǎn)：Hadoop分布式文件系統(tǒng)　出處：《計(jì)算機(jī)工程與設(shè)計(jì)》2017年12期

【摘要】：提出一種基于Spark分布式內(nèi)存計(jì)算框架的遙感大數(shù)據(jù)特征提取策略。采用Landsat8為數(shù)據(jù)源,以計(jì)算歸一化植被指數(shù)(NDVI)、差值植被指數(shù)(DVI)、比值植被指數(shù)(RVI)為例開展實(shí)驗(yàn)。實(shí)驗(yàn)結(jié)果表明,在相同硬件環(huán)境、處理任務(wù)、數(shù)據(jù)量的條件下,Spark處理遙感大數(shù)據(jù)的速度較單機(jī)模式下的處理遙感大數(shù)據(jù)提升了約2倍,基于Hadoop分布式文件系統(tǒng)(HDFS)處理模式較Spark-standalone處理模式處理速度提升了約1.2倍,基于Spark下的HDFS存儲(chǔ)模式下,柵格切分遙感大數(shù)據(jù)較非柵格切分處理速度提高了約1.5倍。
[Abstract]:A remote sensing big data feature extraction strategy based on Spark distributed memory computing framework is proposed.Using Landsat8 as the data source, the experiments were carried out with the examples of the calculation of normalized vegetation index (NDVI), the difference vegetation index (DVI) and the ratio vegetation index (RVI).The experimental results show that the speed of Spark processing remote sensing big data under the same hardware environment, processing task and data volume is about 2 times faster than that of single machine mode.The processing speed of distributed file system (Hadoop) based on Hadoop is about 1.2 times faster than that of Spark-standalone. In HDFS storage mode based on Spark, the processing speed of remote sensing of grid segmentation is about 1.5 times faster than that of non-grid segmentation.
【作者單位】： 新疆大學(xué)軟件學(xué)院;
【基金】：國(guó)家自然科學(xué)基金項(xiàng)目(61562086;61462079;61363083;61262088) 新疆“萬人計(jì)劃”后備基金項(xiàng)目(wr2015bj01) 新疆自治區(qū)研究生科研創(chuàng)新基金項(xiàng)目(XJGRI2016029)
【分類號(hào)】：TP751

【相似文獻(xiàn)】

相關(guān)期刊論文 前3條

1 朱教群,梅炳初,何利萍,陳艷林;Synthesis of Ti_3SiC_2 by spark plasma sintering(SPS) of elemental powders[J];Transactions of Nonferrous Metals Society of China;2003年01期

2 許劍光;張厚安;江國(guó)健;張寶林;李文蘭;;SiC whisker reinforced MoSi_2 composite prepared by spark plasma sintering from COSHS-ed powder[J];Transactions of Nonferrous Metals Society of China;2006年S2期

3 謝衍;陳敬熊;苗俊剛;;K波段微波高光譜輻射計(jì)的晴空定標(biāo)方法[J];北京航空航天大學(xué)學(xué)報(bào);2014年04期

，

本文編號(hào)：1725943