中国移动hadoop数据挖掘平台介绍.ppt

Industry --BigCloudbasedParallelDataMiningPlatformOutlineIntroduceBC-municationCorporation()Subscribers:500millionSubscribers’CDR(callingdatarecord)data5~8TB/pany(>20millionsubscribers)Voice:100million*1KB=100GB/daySMS:100~200million*1KB=100~200GB/day……Networksignalingdata,pany(>20millionsubscribers)GPRSsignalingdata:48GB/panies3Gsignalingdata:300GB/paniesvoice,SMSsignalingdata,……LargeScaleDataApplicationsandcurrentsolutionPrecisionmarketingAnalysisofUserBehaviorCustomerChurnPredictionServiceAssociationAnalysis……workQOSAnalysisSingallingDataAnalysis......ServiceOptimizationandLogProcessingSpamMessageFiltering……Commercialdatabase/mercialDataMiningToolsMostarerunningonUnixServers,datastoredinStorageArraysTheRequirementsCurrentsolutionClemetineEnterpriseMinerIntelligentMinerWhat’sBASSBASS(BusinessAnalysisSupportSystem)tosupportenterprisedecision-making,marketingmanagementanalysis,,dataprocesslayer,datadisplaylayer,applicationlayerMainoperationindataprocesslayeris:Dataextractfromothersystem,DatatransferDatagatherDatastatics…Basedondatabasesystem,mostofoperationaredealindatabase,whichrealizesELT(Extract,LoadandTransfer),,%investisonhardwareBecausethere’sdifferentcritiabetweentheunixserver,whenenlargement,’tsupportaBASS,ineverybranchsubsystme,there’sabout3-5servers,suchasETLServer,DatabaseServer,InterfaceServer,,pany,’tbesupportwellOfflinedatabackup(5branches)costlotsoftime,onlinedatabackup(8branches)costlotsofresource,filebackup(18branches)restoreslowlyWhatistheBC-PDMBC-PDM:BigCloudbasedParallelDataMiningPlatformAdataminingsolutionforlarge-scaledataanalysisMassivescalability-moditymachinesandfreesoftwareCustomization–facingtoapplication