發(fā)布時(shí)間：2018-01-15 04:19

  本文關(guān)鍵詞：海量存儲(chǔ)系統(tǒng)中并行文件系統(tǒng)的測(cè)試與優(yōu)化　出處：《上海交通大學(xué)》2012年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 海量存儲(chǔ) 并行文件系統(tǒng) 預(yù)取 緩存 條帶化

【摘要】：海量存儲(chǔ)系統(tǒng)(Mass Storage System)是為存儲(chǔ)海量數(shù)據(jù)而研制的存儲(chǔ)系統(tǒng),其本質(zhì)特征在于該存儲(chǔ)系統(tǒng)的可擴(kuò)展性,即能在擴(kuò)展系統(tǒng)容量和性能的同時(shí),不增加系統(tǒng)管理的復(fù)雜性。并行文件系統(tǒng)(Parallel File System)是由一組節(jié)點(diǎn)(Node)組成的,這組節(jié)點(diǎn)通過相互之間的通信與協(xié)作,以更快的速度完成一項(xiàng)大規(guī)模的計(jì)算任務(wù)。并行文件系統(tǒng)是構(gòu)筑高性能海量存儲(chǔ)系統(tǒng)的基石,所以無(wú)論是在高性能計(jì)算還是在云存儲(chǔ)都有著重要的研究意義,因此近年來(lái)得到廣泛的關(guān)注和研究。 并行文件系統(tǒng)的測(cè)試,大致包括一致性測(cè)試、功能測(cè)試、壓力測(cè)試、性能測(cè)試、推測(cè)性測(cè)試等,其中性能測(cè)試其中的重點(diǎn)。并行文件系統(tǒng)性能的測(cè)試測(cè)試的指標(biāo)大致有兩個(gè),一個(gè)是吞吐量,衡量大文件I/O讀寫能力;另一個(gè)是IOPS,衡量小文件讀寫能力。 常用優(yōu)化并行文件系統(tǒng)性能的策略有數(shù)據(jù)預(yù)取,數(shù)據(jù)緩存、合并小順序I/O、條帶化等技術(shù)。 緩存指利用緩沖區(qū)保存最近訪問過的文件內(nèi)容,以提高對(duì)文件的訪問效率。預(yù)取是指在實(shí)際使用內(nèi)存時(shí)提前將數(shù)據(jù)讀取到內(nèi)存中,從而提高訪問效率。合并小順序I/O,則是將若干個(gè)小文件的操作聚合成一個(gè)大文件的操作,從而減小磁盤操作的次數(shù)。條帶化是把連續(xù)的數(shù)據(jù)塊分割成相同大小的數(shù)據(jù)塊,將每段數(shù)據(jù)分別寫入到陣列中不同的磁盤。 本文主要研究并行文件系統(tǒng)的測(cè)試方法,以及優(yōu)化性能的策略。研究?jī)?nèi)容主要包括:研究并行文件系統(tǒng)性能測(cè)試的標(biāo)準(zhǔn)及方法、學(xué)習(xí)改進(jìn)文件系統(tǒng)性能的測(cè)試的方法、以GlusterFS為例驗(yàn)證各優(yōu)化策略的效果。實(shí)驗(yàn)證明GlusterFS中的緩存策略改善了超過10MB/s的讀性能,預(yù)取策略的兩個(gè)方法合計(jì)提供了約15MB/s的讀性能。
[Abstract]:Mass Storage system is a storage system developed for storing mass data. Its essential feature lies in the scalability of the storage system. That is, it can expand the capacity and performance of the system at the same time. Parallel file system parallel File system is composed of a set of nodes. This group of nodes completes a large-scale computing task with faster speed through mutual communication and cooperation. Parallel file system is the cornerstone of constructing high performance mass storage system. Therefore, both in high performance computing and cloud storage have important research significance, so in recent years, it has received extensive attention and research. The test of parallel file system includes conformance test, function test, stress test, performance test, conjectural test and so on. Among them, the key points of performance test. There are two indexes of parallel file system performance test, one is throughput, which measures the reading and writing ability of large file I / O; The other is IOPS, which measures the ability to read and write small files. The commonly used strategies to optimize the performance of parallel file systems include data prefetching, data caching, merging small order I / O, striping and so on. Caching is the use of buffers to save the contents of recently accessed files in order to improve the efficiency of accessing files. Prefetching refers to reading data into memory ahead of time when memory is actually used. In order to improve access efficiency, merging small order I / O is the operation of aggregating several small files into one large file. In order to reduce the number of disk operations, striping is to divide the continuous data blocks into blocks of the same size and write each piece of data to a different disk in the array. This paper mainly studies the testing method of parallel file system and the strategy of optimizing performance. The research contents include: research the standard and method of parallel file system performance test. Learn how to improve file system performance testing. GlusterFS is taken as an example to verify the effectiveness of the optimized strategies. Experiments show that the cache policy in GlusterFS improves the reading performance of more than 10MB / s. The two methods of the prefetching strategy together provide about 15 MB / s read performance.
【學(xué)位授予單位】：上海交通大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2012
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前1條

1 白英彩;金崇英;;海量存儲(chǔ)系統(tǒng)的研究與應(yīng)用[J];軟件產(chǎn)業(yè)與工程;2010年05期

，

本文編號(hào)：1426758