本文選題：Hadoop分布式文件系統(tǒng) + 名字節(jié)點(diǎn)��； 參考：《北京郵電大學(xué)》2013年碩士論文

【摘要】：HDFS (Hadoop Distributed File System)是Hadoop的分布式文件系統(tǒng)。Hadoop0.20.203版本中,HDFS采用主從架構(gòu),主要由一個(gè)Namenode,一個(gè)SecondaryNamenode及其眾多的Datanode構(gòu)成。Namenode作為HDFS的單一主服務(wù)器節(jié)點(diǎn),存在單點(diǎn)失效,性能瓶頸,不易擴(kuò)展等缺點(diǎn)。同時(shí),HDFS的設(shè)計(jì)思想主要是通過一些廉價(jià)的主機(jī)和服務(wù)器構(gòu)建一個(gè)分布式的文件存儲(chǔ)集群,硬件失效是常態(tài)。 針對(duì)本系統(tǒng)存在的問題,本文主要進(jìn)行了如下幾方面的工作： 1.介紹了HDFS的基本概念,對(duì)HDFS的發(fā)展歷程,存在的問題,研究現(xiàn)狀進(jìn)行了綜述； 2.詳細(xì)介紹HDFS的系統(tǒng)組件,包括Namenode和Datanode。對(duì)元數(shù)據(jù),數(shù)據(jù)的組織和交互以及數(shù)據(jù)維護(hù)方面進(jìn)行了深入的研究； 3.提出了一種新的分布式Namenode節(jié)點(diǎn)集群方案。Namenode分布式方案主要是將原先Namenode節(jié)點(diǎn)的功能進(jìn)行重新分配。其中Namenode1集群主要用來處理客戶端的請(qǐng)求和管理Datanode節(jié)點(diǎn)的狀態(tài),；Namenode2集群主要用來管理并持久化元數(shù)據(jù)信息以及維護(hù)數(shù)據(jù)節(jié)點(diǎn)與數(shù)據(jù)塊映射信息。Leader節(jié)點(diǎn)主要是轉(zhuǎn)發(fā)客戶端的請(qǐng)求及其監(jiān)控整個(gè)集群運(yùn)行狀態(tài),同時(shí)返回響應(yīng)結(jié)果。對(duì)DRBD, Pacemaker等組件進(jìn)行了深入的研究。認(rèn)真分析了已經(jīng)存在的系統(tǒng)的不足。同時(shí),對(duì)單節(jié)點(diǎn)Namenode進(jìn)行Linux HA的部署驗(yàn)證； 4.介紹了分布式系統(tǒng)常用的數(shù)據(jù)冗余技術(shù)。詳細(xì)研究了HDFS的數(shù)據(jù)冗余機(jī)制,并且對(duì)其進(jìn)行實(shí)驗(yàn)驗(yàn)證。同時(shí),深入研究了數(shù)據(jù)冗余機(jī)制對(duì)數(shù)據(jù)交互以及負(fù)載均衡的影響。 5.總結(jié)全文,提出一些有待改進(jìn)的方面。
[Abstract]:HDFS (Hadoop Distributed File System) is the.Hadoop0.20.203 version of Hadoop's distributed file system. HDFS uses a master-slave architecture, which consists of a single main server node consisting of a Namenode, a SecondaryNamenode and a large number of Datanode.Namenode. There are shortcomings such as single point failure, performance bottleneck, and uneasy expansion. The main idea of HDFS is to build a distributed file storage cluster through cheap hosts and servers, and hardware failure is normal.
In view of the problems existing in the system, this paper mainly focuses on the following aspects:
1. introduces the basic concept of HDFS, summarizes the development process, existing problems and research status of HDFS.
2. detailed introduction of HDFS's system components, including Namenode and Datanode., in-depth research on metadata, data organization and interaction, and data maintenance.
3. a new distributed Namenode node cluster scheme.Namenode distributed scheme is proposed to redistribute the functions of the original Namenode nodes. The Namenode1 cluster is mainly used to deal with the client's request and manage the state of the Datanode node, and the Namenode2 set is used to manage and persist metadata information. And the.Leader node to maintain the data node and the data block mapping information is mainly the request of the forwarding client and the monitoring of the whole cluster running state, and the response results are returned. The components such as DRBD, Pacemaker and other components are deeply studied. The shortcomings of the existing system are carefully analyzed. At the same time, the single node Namenode is carried out in the Linux HA department. Certification;
4. the data redundancy technology used in distributed systems is introduced. The data redundancy mechanism of HDFS is studied in detail, and the experimental verification is carried out. At the same time, the influence of data redundancy mechanism on data interaction and load balancing is deeply studied.
5. summarize the full text, and put forward some aspects to be improved.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2013
【分類號(hào)】：TP333;TP309

【參考文獻(xiàn)】

相關(guān)期刊論文 前3條

1 鞏天寧;周書明;;基于DRBD的Linux高可用集群[J];電腦與信息技術(shù);2012年01期

2 喬鑫;;MooseFS分布式文件系統(tǒng)及應(yīng)用[J];科技浪潮;2009年05期

3 ;SYNCHRONIZATION OF TWO COUPLED HINDMARSH-ROSE NEURONS BY A PACEMAKER[J];Annals of Differential Equations;2011年04期

相關(guān)碩士學(xué)位論文 前2條

1 潘磊穎;多元數(shù)據(jù)服務(wù)器環(huán)境下的元數(shù)據(jù)管理研究[D];華中科技大學(xué);2007年

2 欒亞建;分布式文件系統(tǒng)元數(shù)據(jù)管理研究與優(yōu)化[D];華南理工大學(xué);2010年

，

本文編號(hào)：2023961