1
Big Data: Key Concepts
The three Vs
Big data in general has context in three Vs:
• Sheer quantity of data
• Speed with which data is produced, processed, and digested
• Diversity of sources inside and outside
.
• Fields/Tables
/Columns
• RDBMS/Spreadsheet
• Markers/Tags to
separate elements
• XML/HTML
• No fields/attributes
• Free form text (email body,
notes, articles)
• Audio, video, and image
The different types of data that contribute to this are:
2
Big Data: Key Concepts
An Internet Minute
3
Big Data: Key Concepts
The challenge of the “needle in a haystack”
Separating the signal from the noise1 becomes really relevant
1 http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-≠-information-≠-insights/
4
Big Data: Key Concepts
Macro Trends
Many organizations carry out business based on insights gained from data analysis. There has been
a shift in the size, type, and form of data and in the way data is analyzed.
Data-led Innovation
Data Explosion
• Unstructured data is doubling
every 3 months
• 2011 saw 47% growth overall
• By 2015, number of networked
devices will be 2x global population
• De-coupling data from applications
• Disparate external data shaping
context
• Cost effective mobilization of
massive scale data
Monetization
• Growth of enterprise data
monetization services
• Large retailers monetizing own
data to provide insights to suppliers
Social Media
• Emergence of companies that
scrub and aggregate data from
social media and blogs
• Greater focus on data that provides
insight in a customer’s digital
persona
Technology
• Commodity priced storage and
compute
• Emergence of open source and big
data technologies solving production
problems at scale
Data Mobilization
• Novel approaches to analyze
unstructured data creating shorter
time from data to insight
• Shift towards data consumption in
multiple environments (business
apps, mobile, social)
5
Trends Driving Fundamental Shifts (1/2)
Data volumes are growing, infrastructure is stressed to the breaking point and Big Data offers the
opportunity to address these challenges.
Bringing the Analytics to the DataBringing the Data to the Analytics
• Focus on structuring data for storage
• Serial approach to mobilizing new data
sources
• Episodic analytics
• Pre-defined reports and dashboards
• Data sampling to fine tune algorithms
• Data silos tethered to applications
• Quantitative vs Qualitative Data
• Focus on mobilizing data for analysis
• Immediate ingestion of new data sources
• Continuous data discovery
• Agile, self-service data visualization
• “Data trumps algorithms”
• Data as a platform
• Derive insight from structured and
unstructured data
6
Big Data is the next generation of data warehousing, business analytics and business intelligence. It’s
poised to deliver top line revenues cost efficiently for enterprises based on new technologies (In-
database, MPP, In–memory,…), more agile analysis (runtime, on time,..) and more deep analytics ( new
data mining predictive algorithm, and optimization modeling)
Bring together a large volume and variety of data to find new insights
Structures the data
to answer that
question
Determine what
question to ask
Business IT
IT Business
Traditional Approach
Structured & Repeatable
Analysis
Big Data Approach
Iterative & Exploratory
Analysis
Explores what
questions could be
asked
Delivers a platform
to enable creative
discovery
Trends Driving Fundamental Shifts (2/2)
7
The Business Value of Big Data
The Value Tree
The business value drivers are beginning to follow familiar patterns – more data and better
insights create value
MetricValue Driver
Generate
Revenue
Business
Value
Reduce Cost
Reduce
Working
Capital
Price Price Optimization Variable Margin %
Volume
Sales Force Effectiveness
Product / Distribution
Customer Loyalty
Marketing Campaign
Effectiveness
Sales(-Person)
Demand Forecasting
Accuracy
Net Promoter Score
Campaign
Awareness Level
Product Mix
Mix of Sales from New
Products
New Product
Sales %
Process
Efficiency
Mfg. Cost
R&D Effectiveness
Product Portfolio Optimization/
Product Profitability
Years to First Sales
Margin Dollars
Inventory
Receivables
Inventory Utilization
Risk Management
Inventory Days
on Hand
Annual BenefitImpacted driverAnalytics Capability
Market Pricing Analytics
Sales Force Optimization
Product Availability
Customer Insight Analytics
Marketing Campaign
Analytics
New Product
Launch Analytics
R&D Discovery Analytics
Product Lifecycle Mgmt
Inventory Cover
Credit Risk Analytics
Sales
Sales
Sales
New Product Sales
Sales
Sales
COGS + FPDE
FG + SFG
Inventory
Margin uplift
potential
$41 M - $52 M
Cost reduction
potential
$8 M - $14 M
Working Capital
reduction
potential
$12 M - $17 M
8
The Transformation Journey
Barriers and Myths
There are many barriers to the adoption
of Big Data. Some causes technological
disruptions while others may lead to
certain organization challenges, which
have to be overcome for the seamless
operations.
There are some interesting Big Data
myths that need to be dispelled.
Converging
architectures
Compatibility,
Integration
Data-centricity
Incentives
Sharing and
collaboration
Privacy, liability
Sensitivity
IP
Data science
Visualization
Solution Development
Access and Availability
Ownership
Quality
Data structure &
Architecture (MDM)
9
The Transformation Journey
The Convergent Data Architecture
10
Getting Started
Accenture’s Big Data Discovery service helps organizations identify Big Data opportunities and use
cases that are aligned with business stakeholder needs.
It helps organizations define a delivery road map and an actionable plan with clear business value
delivery goals by phase.
Through Discovery, the team defines a conceptual technical and solution architecture design and helps
to understand the total cost of ownership (TCO) of the technologies chosen.
Approach
The Transformation Journey
Accenture approach
11
The Business Value of Big Data
Impact on different Business Sectors
• Recent research has shown that companies, that use Big Data and analytics to make decisions, are
more productive and make more revenues. Here are some examples of certain business sectors that
utilized Big Data to gain advantage.
.
Emerging Trends with Big Data
• Large-scale clickstream analytics
• Event, location and behavior based
targeting combining social media data
• Sentiment analysis
• Cross-selling, Market Basket Analysis
and Ad targeting
• Deep consumer segmentations
• Merchandizing and Optimization
• Supply-chain management and
analytics
• New services such as price
comparisons or virtual markets
• Operational transparency
Uses:
• Marketing Campaign Analysis
• Sentiment Analysis
• Point of Sale
• Trade surveillance
Emerging Trends with Big Data
• Improve customer experience and
retention
• Tailored real-time recommendations
during customer interactions
• Monetization of data through value
added services
• Enhance operational efficiencies by
detecting infrastructure bottlenecks real-
time
• Network and security analytics, intrusion
detection with a 360 degree view
Uses
• Ad Targeting
• Network Data Analysis
• Search quality
• Data Sandbox
Telecommunication & Media Retail & Consumer Goods
Emerging Trends with Big Data
• Marketing partnerships to develop
enhanced profile of customer
• Targeted offers to cross sell and up-sell
• Performance marketing – improve
promotion effectiveness
• Leverage multiple sources of
unstructured data to improve 360 degree
view of customer
• Customer retention
• Manage credit risks
• Fraud detection and analysis
• Sales force productivity and effectiveness
• Trade portfolio performance and
optimization
Uses
• Risk modeling
• Customer attrition analysis
• Recommendation engine
• Threat analysis (fraud detection)
Financial ServicesFinancial Services
12
The Business Value of Big Data
A real example in Media
Many data sources –
Web Service
Applications, CDN,
Set Top Boxes, DVR
Logs
High Quality of
Service -
Customers receive
Quick resolution of
service issues
MongoDB and
MapReduce
build aggregate views
Session, Loc.,
Timeline, CDN stat
etc.
Administrators see
faults quickly
Rapid alert and
repair
Fewer service
interruptions
1.2 million
recordings/min
Diverse Sources
and formats
Quality of Service is a key business demand for digital TV providers and presents a number of
technical challenges. Tracking infrastructure and client hardware components can generate a range of
unstructured data at huge scope and scale. Aggregate views reveal patterns that enable timely issue
resolution and enable new business opportunities
13
The Business Value of Big Data
A real example in Consumer Goods
14
The Business Value of Big Data
A real example in Financial Services (1/2) – Collecting Social Data
Demographic
s (name,
birthdate)
Contacts
(mobile, email)
Vkontakte Foursquare Facebook
1.7 mln.
clients
1.2 mln.
contact
s
300k profiles
with high
matching
probability
2.5 mln.
profiles
19k
profiles
124k
profiles
Searching
criteria
Available
records
quantity
Profiles found by search criteria
We’ve created custom java tool to:
• search selected social networks for profiles that
matches available client data
• download all publicly available data for these profiles as
it is.
15
The Business Value of Big Data
A real example in Financial Services (2/2) – Outcomes
Opinion leaders
36 Opinion leaders among
Sberbank clients were identified
Average profile of Sberbank client in Foursquare
• Works in center, lives close to city border
• Most shopping is done near living place
• Use underground
• Prefer bars and cafés to restaurants
• Prefer sport entertainment to art
• Prefer parks within city bounds for recreation
16
• Value of Big Data is undisputable because it boost the ability of
data to drive business outcomes aligned to the enterprise. This is
exactly what clients are seeking.
• Seek fact based approaches, look and incorporate business use
cases and data usage patterns to select the right fit-for-purpose
technologies. This is done depending on needs.
• Hybrid solution architectures provide tremendous value, but
there are tradeoffs in system integration and leveraging new
technologies. Initiatives that plan to augment instead of using radical
new approaches are appearing to have more success.
• Enterprise Big Data investments based on looking at the size of
the data maybe wrong. Evaluate speed, variety, and other aspects.
• Executing at an enterprise level to maximize the value of Big
Data requires an aligned strategy. Having an innovative mindset, a
discipline of predictable delivery, and good partners are advantages
Conclusions

uae views on big data

  • 1.
    1 Big Data: KeyConcepts The three Vs Big data in general has context in three Vs: • Sheer quantity of data • Speed with which data is produced, processed, and digested • Diversity of sources inside and outside . • Fields/Tables /Columns • RDBMS/Spreadsheet • Markers/Tags to separate elements • XML/HTML • No fields/attributes • Free form text (email body, notes, articles) • Audio, video, and image The different types of data that contribute to this are:
  • 2.
    2 Big Data: KeyConcepts An Internet Minute
  • 3.
    3 Big Data: KeyConcepts The challenge of the “needle in a haystack” Separating the signal from the noise1 becomes really relevant 1 http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-≠-information-≠-insights/
  • 4.
    4 Big Data: KeyConcepts Macro Trends Many organizations carry out business based on insights gained from data analysis. There has been a shift in the size, type, and form of data and in the way data is analyzed. Data-led Innovation Data Explosion • Unstructured data is doubling every 3 months • 2011 saw 47% growth overall • By 2015, number of networked devices will be 2x global population • De-coupling data from applications • Disparate external data shaping context • Cost effective mobilization of massive scale data Monetization • Growth of enterprise data monetization services • Large retailers monetizing own data to provide insights to suppliers Social Media • Emergence of companies that scrub and aggregate data from social media and blogs • Greater focus on data that provides insight in a customer’s digital persona Technology • Commodity priced storage and compute • Emergence of open source and big data technologies solving production problems at scale Data Mobilization • Novel approaches to analyze unstructured data creating shorter time from data to insight • Shift towards data consumption in multiple environments (business apps, mobile, social)
  • 5.
    5 Trends Driving FundamentalShifts (1/2) Data volumes are growing, infrastructure is stressed to the breaking point and Big Data offers the opportunity to address these challenges. Bringing the Analytics to the DataBringing the Data to the Analytics • Focus on structuring data for storage • Serial approach to mobilizing new data sources • Episodic analytics • Pre-defined reports and dashboards • Data sampling to fine tune algorithms • Data silos tethered to applications • Quantitative vs Qualitative Data • Focus on mobilizing data for analysis • Immediate ingestion of new data sources • Continuous data discovery • Agile, self-service data visualization • “Data trumps algorithms” • Data as a platform • Derive insight from structured and unstructured data
  • 6.
    6 Big Data isthe next generation of data warehousing, business analytics and business intelligence. It’s poised to deliver top line revenues cost efficiently for enterprises based on new technologies (In- database, MPP, In–memory,…), more agile analysis (runtime, on time,..) and more deep analytics ( new data mining predictive algorithm, and optimization modeling) Bring together a large volume and variety of data to find new insights Structures the data to answer that question Determine what question to ask Business IT IT Business Traditional Approach Structured & Repeatable Analysis Big Data Approach Iterative & Exploratory Analysis Explores what questions could be asked Delivers a platform to enable creative discovery Trends Driving Fundamental Shifts (2/2)
  • 7.
    7 The Business Valueof Big Data The Value Tree The business value drivers are beginning to follow familiar patterns – more data and better insights create value MetricValue Driver Generate Revenue Business Value Reduce Cost Reduce Working Capital Price Price Optimization Variable Margin % Volume Sales Force Effectiveness Product / Distribution Customer Loyalty Marketing Campaign Effectiveness Sales(-Person) Demand Forecasting Accuracy Net Promoter Score Campaign Awareness Level Product Mix Mix of Sales from New Products New Product Sales % Process Efficiency Mfg. Cost R&D Effectiveness Product Portfolio Optimization/ Product Profitability Years to First Sales Margin Dollars Inventory Receivables Inventory Utilization Risk Management Inventory Days on Hand Annual BenefitImpacted driverAnalytics Capability Market Pricing Analytics Sales Force Optimization Product Availability Customer Insight Analytics Marketing Campaign Analytics New Product Launch Analytics R&D Discovery Analytics Product Lifecycle Mgmt Inventory Cover Credit Risk Analytics Sales Sales Sales New Product Sales Sales Sales COGS + FPDE FG + SFG Inventory Margin uplift potential $41 M - $52 M Cost reduction potential $8 M - $14 M Working Capital reduction potential $12 M - $17 M
  • 8.
    8 The Transformation Journey Barriersand Myths There are many barriers to the adoption of Big Data. Some causes technological disruptions while others may lead to certain organization challenges, which have to be overcome for the seamless operations. There are some interesting Big Data myths that need to be dispelled. Converging architectures Compatibility, Integration Data-centricity Incentives Sharing and collaboration Privacy, liability Sensitivity IP Data science Visualization Solution Development Access and Availability Ownership Quality Data structure & Architecture (MDM)
  • 9.
    9 The Transformation Journey TheConvergent Data Architecture
  • 10.
    10 Getting Started Accenture’s BigData Discovery service helps organizations identify Big Data opportunities and use cases that are aligned with business stakeholder needs. It helps organizations define a delivery road map and an actionable plan with clear business value delivery goals by phase. Through Discovery, the team defines a conceptual technical and solution architecture design and helps to understand the total cost of ownership (TCO) of the technologies chosen. Approach The Transformation Journey Accenture approach
  • 11.
    11 The Business Valueof Big Data Impact on different Business Sectors • Recent research has shown that companies, that use Big Data and analytics to make decisions, are more productive and make more revenues. Here are some examples of certain business sectors that utilized Big Data to gain advantage. . Emerging Trends with Big Data • Large-scale clickstream analytics • Event, location and behavior based targeting combining social media data • Sentiment analysis • Cross-selling, Market Basket Analysis and Ad targeting • Deep consumer segmentations • Merchandizing and Optimization • Supply-chain management and analytics • New services such as price comparisons or virtual markets • Operational transparency Uses: • Marketing Campaign Analysis • Sentiment Analysis • Point of Sale • Trade surveillance Emerging Trends with Big Data • Improve customer experience and retention • Tailored real-time recommendations during customer interactions • Monetization of data through value added services • Enhance operational efficiencies by detecting infrastructure bottlenecks real- time • Network and security analytics, intrusion detection with a 360 degree view Uses • Ad Targeting • Network Data Analysis • Search quality • Data Sandbox Telecommunication & Media Retail & Consumer Goods Emerging Trends with Big Data • Marketing partnerships to develop enhanced profile of customer • Targeted offers to cross sell and up-sell • Performance marketing – improve promotion effectiveness • Leverage multiple sources of unstructured data to improve 360 degree view of customer • Customer retention • Manage credit risks • Fraud detection and analysis • Sales force productivity and effectiveness • Trade portfolio performance and optimization Uses • Risk modeling • Customer attrition analysis • Recommendation engine • Threat analysis (fraud detection) Financial ServicesFinancial Services
  • 12.
    12 The Business Valueof Big Data A real example in Media Many data sources – Web Service Applications, CDN, Set Top Boxes, DVR Logs High Quality of Service - Customers receive Quick resolution of service issues MongoDB and MapReduce build aggregate views Session, Loc., Timeline, CDN stat etc. Administrators see faults quickly Rapid alert and repair Fewer service interruptions 1.2 million recordings/min Diverse Sources and formats Quality of Service is a key business demand for digital TV providers and presents a number of technical challenges. Tracking infrastructure and client hardware components can generate a range of unstructured data at huge scope and scale. Aggregate views reveal patterns that enable timely issue resolution and enable new business opportunities
  • 13.
    13 The Business Valueof Big Data A real example in Consumer Goods
  • 14.
    14 The Business Valueof Big Data A real example in Financial Services (1/2) – Collecting Social Data Demographic s (name, birthdate) Contacts (mobile, email) Vkontakte Foursquare Facebook 1.7 mln. clients 1.2 mln. contact s 300k profiles with high matching probability 2.5 mln. profiles 19k profiles 124k profiles Searching criteria Available records quantity Profiles found by search criteria We’ve created custom java tool to: • search selected social networks for profiles that matches available client data • download all publicly available data for these profiles as it is.
  • 15.
    15 The Business Valueof Big Data A real example in Financial Services (2/2) – Outcomes Opinion leaders 36 Opinion leaders among Sberbank clients were identified Average profile of Sberbank client in Foursquare • Works in center, lives close to city border • Most shopping is done near living place • Use underground • Prefer bars and cafés to restaurants • Prefer sport entertainment to art • Prefer parks within city bounds for recreation
  • 16.
    16 • Value ofBig Data is undisputable because it boost the ability of data to drive business outcomes aligned to the enterprise. This is exactly what clients are seeking. • Seek fact based approaches, look and incorporate business use cases and data usage patterns to select the right fit-for-purpose technologies. This is done depending on needs. • Hybrid solution architectures provide tremendous value, but there are tradeoffs in system integration and leveraging new technologies. Initiatives that plan to augment instead of using radical new approaches are appearing to have more success. • Enterprise Big Data investments based on looking at the size of the data maybe wrong. Evaluate speed, variety, and other aspects. • Executing at an enterprise level to maximize the value of Big Data requires an aligned strategy. Having an innovative mindset, a discipline of predictable delivery, and good partners are advantages Conclusions