本文選題：電子商務(wù) + 數(shù)據(jù)采集　； 參考：《東南大學(xué)》2016年碩士論文

【摘要】：隨著電子商務(wù)的發(fā)展,電子商務(wù)系統(tǒng)變得越來越復(fù)雜�，F(xiàn)在的電子商務(wù)系統(tǒng)不但需要支持各種各樣的商品展示、購買和配送,還需要處理電商大促時產(chǎn)生的巨大交易流量,為了滿足這個需求,電子商務(wù)系統(tǒng)需要由許多運營系統(tǒng)來支持。一次成功的購物需要各個系統(tǒng)之間相互配合完成,而這期間任何一個系統(tǒng)出現(xiàn)問題都會導(dǎo)致購物的失敗,而且問題的定位和排查也非常困難。為了使得在復(fù)雜的運營平臺下高效、快捷地定位和排查問題,本文分析了電商業(yè)務(wù)追蹤系統(tǒng)的需求,設(shè)計并實現(xiàn)了面向電商運營平臺的業(yè)務(wù)追蹤系統(tǒng)。通過對日志數(shù)據(jù)的采集和處理,本系統(tǒng)能夠再現(xiàn)整個業(yè)務(wù)的執(zhí)行流程和進(jìn)度,便于問題的定位和排查,同時,根據(jù)數(shù)據(jù)聚類結(jié)果與歷史數(shù)據(jù)的比較結(jié)果來實現(xiàn)業(yè)務(wù)風(fēng)險預(yù)警。本文主要工作歸納如下：(1)提出了面向電商運營平臺的業(yè)務(wù)追蹤系統(tǒng)的設(shè)計方案。根據(jù)系統(tǒng)的輸入輸出,對系統(tǒng)的需求進(jìn)行了分析,梳理了系統(tǒng)的主要功能,設(shè)計了業(yè)務(wù)追蹤系統(tǒng)的總體架構(gòu)和數(shù)據(jù)庫表結(jié)構(gòu)。(2)提出了電商業(yè)務(wù)系統(tǒng)中日志數(shù)據(jù)的采集與處理方法,包括日志數(shù)據(jù)的讀取、篩選、切分、處理和保存方案,其中采用Sunday字符串匹配算法來篩選日志,使用布隆過濾器來進(jìn)行日志去重,采用分段生成唯一序號的方法來生成本系統(tǒng)的數(shù)據(jù)庫主鍵等。(3)提出了電商業(yè)務(wù)追蹤與分析方法,包括電商業(yè)務(wù)追蹤和分析過程中的數(shù)據(jù)查詢、排序、聚類和預(yù)警方案,其中采用并發(fā)查詢和排序進(jìn)行日志的追蹤,使用聚類算法對數(shù)據(jù)進(jìn)行聚類,通過與歷史數(shù)據(jù)對比來進(jìn)行業(yè)務(wù)預(yù)警等。(4)實現(xiàn)了電商業(yè)務(wù)追蹤系統(tǒng),并進(jìn)行了系統(tǒng)測試,現(xiàn)已在企業(yè)真實的場景下進(jìn)行了部署和使用。
[Abstract]:With the development of e-commerce, e-commerce system becomes more and more complex. Today's e-commerce systems not only need to support the display, purchase and distribution of all kinds of goods, but also need to deal with the huge volume of transactions generated by the promotion of e-commerce, in order to meet this demand. E-commerce systems need to be supported by many operating systems. A successful shopping requires the cooperation between the various systems, and during this period any system problems will lead to the failure of shopping, and the problem location and troubleshooting is very difficult. In order to make the problem of locating and checking efficiently and quickly under the complex operation platform, this paper analyzes the requirements of the e-commerce business tracking system, and designs and implements a business tracking system oriented to the e-commerce operation platform. Through the collection and processing of log data, the system can reproduce the whole business execution process and progress, easy to locate and search problems, at the same time, according to the data clustering results and historical data comparison results to achieve business risk early warning. The main work of this paper is summarized as follows: (1) the design of business tracking system for e-commerce platform is proposed. According to the input and output of the system, the requirements of the system are analyzed, and the main functions of the system are sorted out. The architecture of the service tracking system and the structure of the database table are designed. (2) the methods of collecting and processing the log data in the e-commerce business system are proposed, including the reading, filtering, splitting, processing and saving of the log data. Among them, Sunday string matching algorithm is used to filter log, Bron filter is used to remove log, and the method of creating unique sequence number is used to generate the database primary key of the system. (3) the method of tracking and analyzing e-commerce services is proposed. It includes data query, sorting, clustering and early warning scheme in the process of business tracing and analysis, in which concurrent query and sorting are used for log tracing, and clustering algorithm is used to cluster data. Compared with the historical data to carry on the business early warning and so on. (4) implemented the electronic commerce service tracking system, and carried on the system test, has now carried on the deployment and the use in the enterprise real scene.
【學(xué)位授予單位】：東南大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2016
【分類號】：TP311.52

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 倪珊珊;張璇;李彤;張瑞云;;面向方面業(yè)務(wù)流程建模中的方面追蹤[J];計算機(jī)科學(xué);2015年08期

2 陸婷;房俊;喬彥克;;基于HBase的交通流數(shù)據(jù)實時存儲系統(tǒng)[J];計算機(jī)應(yīng)用;2015年01期

3 張鵬;李鵬霄;任彥;林海倫;楊嶸;鄭超;;面向大數(shù)據(jù)的分布式流處理技術(shù)綜述[J];計算機(jī)研究與發(fā)展;2014年S2期

4 劉子英;唐宏建;肖嘉耀;張騫;;基于流式計算的Web實時故障診斷分析與設(shè)計[J];華東交通大學(xué)學(xué)報;2014年01期

5 強(qiáng)彥;盧軍佐;劉濤;裴博;;基于HBase的并行BFS方法[J];計算機(jī)科學(xué);2013年03期

6 張榆;馬友忠;孟小峰;;一種基于HBase的高效空間關(guān)鍵字查詢策略[J];小型微型計算機(jī)系統(tǒng);2012年10期

7 袁佳琳;高建華;;基于工作量和錯誤日志的Web應(yīng)用可靠性度量方法[J];計算機(jī)應(yīng)用與軟件;2012年10期

8 王曉平;;基于業(yè)務(wù)事件上下文感知的物流跟蹤系統(tǒng)研究[J];物流技術(shù);2010年21期

9 張生華;秦拯;宋勇;張忠志;;基于值域二次哈希方法的布魯姆過濾器[J];微計算機(jī)信息;2010年06期

10 王保山;毛保華;杜鵬;劉海東;;快遞公司信息追蹤系統(tǒng)研究[J];物流技術(shù);2009年11期

，

本文編號：2098730