本文選題：Hadoop分布式文件系統(tǒng) + 名字節(jié)點�。� 參考：《北京郵電大學》2013年碩士論文

【摘要】：HDFS (Hadoop Distributed File System)是Hadoop的分布式文件系統(tǒng)。Hadoop0.20.203版本中,HDFS采用主從架構,主要由一個Namenode,一個SecondaryNamenode及其眾多的Datanode構成。Namenode作為HDFS的單一主服務器節(jié)點,存在單點失效,性能瓶頸,不易擴展等缺點。同時,HDFS的設計思想主要是通過一些廉價的主機和服務器構建一個分布式的文件存儲集群,硬件失效是常態(tài)。 針對本系統(tǒng)存在的問題,本文主要進行了如下幾方面的工作： 1.介紹了HDFS的基本概念,對HDFS的發(fā)展歷程,存在的問題,研究現(xiàn)狀進行了綜述； 2.詳細介紹HDFS的系統(tǒng)組件,包括Namenode和Datanode。對元數(shù)據(jù),數(shù)據(jù)的組織和交互以及數(shù)據(jù)維護方面進行了深入的研究； 3.提出了一種新的分布式Namenode節(jié)點集群方案。Namenode分布式方案主要是將原先Namenode節(jié)點的功能進行重新分配。其中Namenode1集群主要用來處理客戶端的請求和管理Datanode節(jié)點的狀態(tài),；Namenode2集群主要用來管理并持久化元數(shù)據(jù)信息以及維護數(shù)據(jù)節(jié)點與數(shù)據(jù)塊映射信息。Leader節(jié)點主要是轉發(fā)客戶端的請求及其監(jiān)控整個集群運行狀態(tài),同時返回響應結果。對DRBD, Pacemaker等組件進行了深入的研究。認真分析了已經(jīng)存在的系統(tǒng)的不足。同時,對單節(jié)點Namenode進行Linux HA的部署驗證； 4.介紹了分布式系統(tǒng)常用的數(shù)據(jù)冗余技術。詳細研究了HDFS的數(shù)據(jù)冗余機制,并且對其進行實驗驗證。同時,深入研究了數(shù)據(jù)冗余機制對數(shù)據(jù)交互以及負載均衡的影響。 5.總結全文,提出一些有待改進的方面。
[Abstract]:HDFS (Hadoop Distributed File System) is the.Hadoop0.20.203 version of Hadoop's distributed file system. HDFS uses a master-slave architecture, which consists of a single main server node consisting of a Namenode, a SecondaryNamenode and a large number of Datanode.Namenode. There are shortcomings such as single point failure, performance bottleneck, and uneasy expansion. The main idea of HDFS is to build a distributed file storage cluster through cheap hosts and servers, and hardware failure is normal.
In view of the problems existing in the system, this paper mainly focuses on the following aspects:
1. introduces the basic concept of HDFS, summarizes the development process, existing problems and research status of HDFS.
2. detailed introduction of HDFS's system components, including Namenode and Datanode., in-depth research on metadata, data organization and interaction, and data maintenance.
3. a new distributed Namenode node cluster scheme.Namenode distributed scheme is proposed to redistribute the functions of the original Namenode nodes. The Namenode1 cluster is mainly used to deal with the client's request and manage the state of the Datanode node, and the Namenode2 set is used to manage and persist metadata information. And the.Leader node to maintain the data node and the data block mapping information is mainly the request of the forwarding client and the monitoring of the whole cluster running state, and the response results are returned. The components such as DRBD, Pacemaker and other components are deeply studied. The shortcomings of the existing system are carefully analyzed. At the same time, the single node Namenode is carried out in the Linux HA department. Certification;
4. the data redundancy technology used in distributed systems is introduced. The data redundancy mechanism of HDFS is studied in detail, and the experimental verification is carried out. At the same time, the influence of data redundancy mechanism on data interaction and load balancing is deeply studied.
5. summarize the full text, and put forward some aspects to be improved.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333;TP309

【參考文獻】

相關期刊論文 前3條

1 鞏天寧;周書明;;基于DRBD的Linux高可用集群[J];電腦與信息技術;2012年01期

2 喬鑫;;MooseFS分布式文件系統(tǒng)及應用[J];科技浪潮;2009年05期

3 ;SYNCHRONIZATION OF TWO COUPLED HINDMARSH-ROSE NEURONS BY A PACEMAKER[J];Annals of Differential Equations;2011年04期

相關碩士學位論文 前2條

1 潘磊穎;多元數(shù)據(jù)服務器環(huán)境下的元數(shù)據(jù)管理研究[D];華中科技大學;2007年

2 欒亞建;分布式文件系統(tǒng)元數(shù)據(jù)管理研究與優(yōu)化[D];華南理工大學;2010年

，

本文編號：2023961