一種適用于P2P-VoD業(yè)務的分布式存儲系統(tǒng)
發(fā)布時間：2018-07-23 11:57
【摘要】：近年來,在線網(wǎng)絡(luò)視頻業(yè)務已逐步發(fā)展為網(wǎng)絡(luò)音樂、網(wǎng)絡(luò)新聞和即時通信以外的第四大類互聯(lián)網(wǎng)應用,成為人們獲取電影、電視、視頻等數(shù)字內(nèi)容的重要媒介。P2P視頻點播業(yè)務作為目前主流的網(wǎng)絡(luò)視頻服務之一,擁有大規(guī)模的用戶群體。隨著互聯(lián)網(wǎng)基礎(chǔ)設(shè)施的大范圍普及和影視媒體琳瑯滿目的發(fā)展,越來越多的用戶選擇使用視頻點播服務來滿足自己獲取信息和娛樂的需求。然而,用戶數(shù)量的不斷增多和用戶對熱門節(jié)目需求的迅速增長同時也為在線網(wǎng)絡(luò)視頻服務提出了更加嚴格的要求。 媒體數(shù)據(jù)量的急劇增長為視頻點播系統(tǒng)帶來了新的問題。首先,海量的媒體數(shù)據(jù)存儲為視頻點播系統(tǒng)帶來了巨大的硬件開銷；其次,為了維持系統(tǒng)的正常運轉(zhuǎn),系統(tǒng)的可用性及數(shù)據(jù)的可靠性需要得到保障；最后,為了滿足大量用戶同時使用系統(tǒng)服務,同時使用戶獲得更流暢的播放體驗,需要盡可能的降低系統(tǒng)負載,提高系統(tǒng)性能。 P2P視頻點播系統(tǒng)中的媒體源服務器負責存儲系統(tǒng)中所有節(jié)目的完整數(shù)據(jù)。短時間內(nèi)大量并發(fā)的數(shù)據(jù)請求對媒體源服務器造成巨大的壓力,導致系統(tǒng)負載過高,資源占用率急劇升高,處理能力下降。另外,一些突發(fā)性的系統(tǒng)故障可能導致媒體源服務器宕機、系統(tǒng)崩潰甚至數(shù)據(jù)丟失,影響整個系統(tǒng)的正常運行,最終導致視頻播放不流暢,用戶體驗下降。 隨著信息服務的日益深化,越來越多的數(shù)據(jù)要求進行長期的低成本存儲,海量數(shù)據(jù)的存儲和處理已成為當前數(shù)據(jù)中心面臨的首要問題,數(shù)據(jù)的分布式存儲及云計算技術(shù)迅速成為IT領(lǐng)域的研究熱點。Hadoop分布式文件系統(tǒng)(HDFS)是Apache Hadoop的基礎(chǔ)文件系統(tǒng),它能夠使用廉價的PC機集群構(gòu)建的系統(tǒng)為海量數(shù)據(jù)提供高可靠性、高容錯性的保障,同時能夠為數(shù)據(jù)密集型應用提供高吞吐量的數(shù)據(jù)訪問,非常適合于大規(guī)模數(shù)據(jù)集長期的低成本存儲。 本文針對P2P視頻點播系統(tǒng)中媒體源服務器的種種問題,提出一項新的解決方案—基于HDFS的適用于P2P-VoD業(yè)務的分布式存儲系統(tǒng)。該分布式存儲系統(tǒng)利用HDFS的可擴展性及可靠性特點,同時在HDFS的數(shù)據(jù)節(jié)點上新增一項本地緩存的功能以改進系統(tǒng)負載性能。通過搭建真實實驗環(huán)境,使用多種測試方法從功能和性能上驗證了該系統(tǒng)的可用性、可靠性,同時驗證了改進后的媒體源服務器在系統(tǒng)負載性能上得到了很大提升。
[Abstract]:In recent years, the online network video business has gradually developed into the fourth largest category of Internet applications other than network music, network news and instant messaging. As one of the mainstream network video services, P2P video-on-demand (VOD), an important medium of digital content such as video, has a large number of users. With the wide spread of Internet infrastructure and the development of video media, more and more users choose to use VOD services to meet their needs for information and entertainment. However, the increasing number of users and the rapid growth of users' demand for popular programs also put forward more stringent requirements for online video services. The rapid growth of media data has brought new problems to VOD system. First, mass media data storage brings huge hardware overhead for VOD system. Secondly, in order to maintain the normal operation of the system, the availability of the system and the reliability of the data need to be guaranteed. In order to satisfy a large number of users using system services at the same time, while enabling users to get a smoother playback experience, it is necessary to minimize the system load as much as possible. The media source server in P2P VOD system is responsible for storing the complete data of all programs in the system. A large number of concurrent data requests in a short period of time cause tremendous pressure on the media source server, resulting in excessive system load, a sharp increase in resource occupancy, and a decline in processing power. In addition, some sudden system failures may lead to media source server downtime, system crash or even data loss, affecting the normal operation of the entire system, resulting in the video playback is not smooth, user experience decline. With the deepening of information services, more and more data require long-term low-cost storage, mass data storage and processing has become the current data center facing the primary problem. Hadoop distributed file system (HDFS) is the basic file system of Apache Hadoop. It can provide high reliability for mass data by using cheap PC cluster system. The guarantee of high fault tolerance and the ability to provide high throughput data access for data-intensive applications are very suitable for the long-term low-cost storage of large data sets. Aiming at the problems of media source server in P2P VOD system, this paper presents a new solution, a distributed storage system based on HDFS for P2P-VoD services. The distributed storage system takes advantage of the extensibility and reliability of HDFS and adds a local cache to the data node of HDFS to improve the load performance of the system. The availability and reliability of the system are verified by using a variety of test methods in a real experimental environment. At the same time, it is verified that the improved media source server has greatly improved the load performance of the system.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP333
[Abstract]:In recent years, the online network video business has gradually developed into the fourth largest category of Internet applications other than network music, network news and instant messaging. As one of the mainstream network video services, P2P video-on-demand (VOD), an important medium of digital content such as video, has a large number of users. With the wide spread of Internet infrastructure and the development of video media, more and more users choose to use VOD services to meet their needs for information and entertainment. However, the increasing number of users and the rapid growth of users' demand for popular programs also put forward more stringent requirements for online video services. The rapid growth of media data has brought new problems to VOD system. First, mass media data storage brings huge hardware overhead for VOD system. Secondly, in order to maintain the normal operation of the system, the availability of the system and the reliability of the data need to be guaranteed. In order to satisfy a large number of users using system services at the same time, while enabling users to get a smoother playback experience, it is necessary to minimize the system load as much as possible. The media source server in P2P VOD system is responsible for storing the complete data of all programs in the system. A large number of concurrent data requests in a short period of time cause tremendous pressure on the media source server, resulting in excessive system load, a sharp increase in resource occupancy, and a decline in processing power. In addition, some sudden system failures may lead to media source server downtime, system crash or even data loss, affecting the normal operation of the entire system, resulting in the video playback is not smooth, user experience decline. With the deepening of information services, more and more data require long-term low-cost storage, mass data storage and processing has become the current data center facing the primary problem. Hadoop distributed file system (HDFS) is the basic file system of Apache Hadoop. It can provide high reliability for mass data by using cheap PC cluster system. The guarantee of high fault tolerance and the ability to provide high throughput data access for data-intensive applications are very suitable for the long-term low-cost storage of large data sets. Aiming at the problems of media source server in P2P VOD system, this paper presents a new solution, a distributed storage system based on HDFS for P2P-VoD services. The distributed storage system takes advantage of the extensibility and reliability of HDFS and adds a local cache to the data node of HDFS to improve the load performance of the system. The availability and reliability of the system are verified by using a variety of test methods in a real experimental environment. At the same time, it is verified that the improved media source server has greatly improved the load performance of the system.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP333
【共引文獻】
相關(guān)期刊論文 前3條
1 陳卓;馮鋼;周江;;分層P2P點播系統(tǒng)中優(yōu)化的帶寬資源分配策略[J];電子科技大學學報;2014年03期
2 劉春陽;張鐵贏;陳國耀;余智華;;基于多任務下載的P2P-VoD系統(tǒng)的測量與分析[J];計算機科學與探索;2014年11期
3 陳靜;周兵;任晨珊;董小社;;分層P2P實時流媒體系統(tǒng)中基于演進博弈的超級組對等節(jié)點選擇算法[J];通信學報;2013年12期
相關(guān)博士學位論文 前5條
1 趙揚;基于用戶行為分析的視頻點播系統(tǒng)優(yōu)化技術(shù)研究[D];中國科學技術(shù)大學;2014年
2 樊沛;虛擬計算環(huán)境中面向通信特征的應用部署優(yōu)化技術(shù)研究[D];國防科學技術(shù)大學;2012年
3 劉婷婷;基于WMN的P2P流媒體組播若干關(guān)鍵技術(shù)研究[D];北京交通大學;2014年
4 徐嬴穎;云下載系統(tǒng)的理論模型與存儲資源分配算法研究[D];北京交通大學;2014年
5 叢鑫;云輔助P2P-VoD系統(tǒng)服務器帶寬資源優(yōu)化關(guān)鍵技術(shù)研究[D];北京郵電大學;2013年
相關(guān)碩士學位論文 前9條
1 張華;一種基于社會網(wǎng)絡(luò)特性的P2P視頻分享系統(tǒng)研究與仿真[D];重慶理工大學;2013年
2 龔光敏;一種P2P視頻點播系統(tǒng)的日志分析子系統(tǒng)設(shè)計與實現(xiàn)[D];華中科技大學;2013年
3 張小彪;基于網(wǎng)絡(luò)編碼的P2P點播算法設(shè)計與仿真[D];華中科技大學;2013年
4 張U，
本文編號：2139337
本文鏈接：http://sikaile.net/kejilunwen/jisuanjikexuelunwen/2139337.html
最近更新
教材專著
