本文關鍵詞：面向油田科技項目管理的大數據查詢優(yōu)化研究　出處：《東北石油大學》2017年碩士論文　論文類型：學位論文

  更多相關文章： 科技項目管理 大數據 ETL設計 查詢優(yōu)化

【摘要】：2009年大慶油田科技管理平臺開始上線運行,平臺實現(xiàn)了對科技項目審核的全生命周期管理。隨著時代的變遷,有關油田科技項目的數據量呈逐年增長態(tài)勢,龐大的數據量給科技項目管理增加了諸多龐雜的工作,為科技項目的高效管理和避免重復立項等情況的發(fā)生,如何能夠快速和正確的判斷別相似的海量數據信息是要解決的首要問題。因此,本課題擬引入大數據的相關概念,通過搭建大數據運行環(huán)境,應用大數據的相關技術對油田科技項目的查詢進行優(yōu)化研究,以此來提高油田項目管理的效率。本文基于課題研究的背景及實際需求,首先,構建了Hadoop大數據運行環(huán)境,針對油田科技項目的數據類型和數據源,展開了存儲結構的研究和搭建,提供了對異地數據源的數據進行加載存儲的策略,以及針對多種類型數據中的結構化數據展開研究,設計支持向存儲后的數據進行ETL處理的NoSQL數據庫。其次,在此基礎上建立基于Hadoop平臺的ETL體系結構,實現(xiàn)更為合理的數據ETL工作流程,為下一步的查詢優(yōu)化提供保證。最后,針對油田科技項目管理的數據查詢,結合當前大數據查詢方法的弊端,分析并研究其底層運行機制和流程,展開優(yōu)化設計,從而提高整個大數據查詢系統(tǒng)的效率,較好的解決了油田科技項目在面向海量數據時的管理效率問題。本課題針對油田科技項目管理平臺的實際發(fā)展及需要,將基于大數據環(huán)境下的存儲結構,查詢方法融合于油田科技項目管理上,并開發(fā)了相應的系統(tǒng)進行實驗,驗證其可行性,為課題的實現(xiàn)提供了理論支撐和實例驗證,在一定程度上對數據量日益增長的油田科技項目管理具有一定的理論意義與參考價值。
[Abstract]:In 2009, Daqing oilfield science and technology management platform began to run online, the platform realized the full life cycle management of science and technology project audit. The volume of scientific and technological projects in oilfields is increasing year by year, and the huge amount of data adds a lot of complicated work to the management of scientific and technological projects, which is the occurrence of the efficient management of scientific and technological projects and the avoidance of duplicate projects. How to quickly and correctly judge the similar mass of data information is the first problem to be solved. Therefore, this paper intends to introduce the relevant concepts of big data, through the establishment of big data operating environment. In order to improve the efficiency of oil field project management, big data's related technology is used to optimize the query of oilfield science and technology project. Firstly, based on the background of the research and the actual needs. The Hadoop big data environment is constructed, and the storage structure is studied and built according to the data types and data sources of oilfield science and technology projects. The strategy of loading and storing data from remote data sources and the research of structured data in various types of data are provided. Design the NoSQL database which supports the ETL processing to the stored data. Secondly, build the ETL architecture based on the Hadoop platform. To achieve a more reasonable data ETL workflow, for the next step of query optimization to provide assurance. Finally, for oilfield science and technology project management data query, combined with the current shortcomings of big data query method. Analysis and study of its underlying operation mechanism and process, the development of optimization design, so as to improve the efficiency of the whole big data query system. Better solve the problem of management efficiency of oilfield science and technology projects in the face of massive data. This subject will be based on the storage structure of big data environment in view of the actual development and needs of oilfield science and technology project management platform. The query method is integrated into the oil field science and technology project management, and the corresponding system is developed to carry on the experiment, to verify its feasibility, provides the theory support and the example verification for the realization of the subject. To a certain extent, it has certain theoretical significance and reference value for the management of oil field science and technology project.
【學位授予單位】：東北石油大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TE4;TP311.13

【相似文獻】

中國期刊全文數據庫 前10條

1 岳耀書，蔣誕一，向新柱;確定科技項目的基本模式研究[J];武漢紡織工學院學報;1994年S1期

2 趙明;在科技項目中的應用加強項目管理[J];石油規(guī)劃設計;2002年02期

3 李家慶;孫宴增;盧明;關新;吳兵;;科技項目管理更要科學化[J];中國石油企業(yè);2006年11期

4 胡克松;;石油企業(yè)科技項目管理存在的問題及對策研究[J];經濟研究導刊;2012年12期

5 帥訓波;李家慶;葛蘭;霍思，

本文編號：1396196