Big Data LDN 2018: LESSONS LEARNED FROM DEPLOYING REAL-WORLD AI SYSTEMS

TOP 5 LESSONS
LEARNED
IN DEPLOYING AI IN THE
REAL WORLD

© 2018 PURE STORAGE INC.2
QUESTION ON EVERYONE’S MIND:
WHY IS A STORAGE COMPANY TALKING
ABOUT AI?

NEW ALGORITHMS
Massively Parallel Delivering
Superhuman Accuracy
MODERN COMPUTE
Massively Parallel Architecture
Driving Performance
GPU- THOUSANDS OF CORES
BIG DATA
Data is the New Oil
50 Zettabytes Created in 2020
EXPLOSION IN ARTIFICIAL INTELLIGENCE
FUELED BY PARALLEL COMPUTE, NEW ALGORITHMS, AND BIG DATA

FRAMEWORKS GPU SERVER STORAGE
TECHNOLOGIES OF THE BIG BANG
WHAT CUSTOMERS DEPLOY

DATA IS VITAL TO MACHINE LEARNING
OBSERVATION BY PROF. ANDREW NG, AI LUMINARY

“We don’t have better algorithms,
we just have more data”
PETER NORVIG
Engineering Director, Google

The AI “hierarchy of needs”
credit: Monica Rogati
ML algorithms: linear & logistic
regression, k-means clustering, decision
trees, etc.
Validation: A/B testing, detecting model
drift over time✓
Data preparation: cleaning, feature
identification, exploration, etc.
Data acquisition: ingest, transformation,
and representation of data for analysis

TOP 5 LESSONS LEARNED
1.AIisaDataPipeline

WHAT MOST THINK IS AI
NEW POSSIBILITIES
For Nearly Every Industry
FRAMEWORKS
To Get Started
GPU
The Engine

AI IS SO MUCH MORE
“Hidden Technical Debt in Machine Learning Systems”, Google NIPS 2015

COMPLEXITIES OF AI IN PRODUCTION
INGEST
From sensors, machines,
& user generated
CLEAN &
TRANSFORM
Label, anomaly detection, ETL,
prep, stage
EXPLORE
Quickly iterate to
converge on models
TRAIN
Run for hours to days in
production cluster
CPU Servers GPU Server GPU Production Cluster
COPY &
TRANSFORM
COPY &
TRANSFORM
COPY &
TRANSFORM

WIDE RANGE OF NEEDS IN AI PIPELINE
SIGNIFICANT CHALLENGE TO LEGACY STORAGE
INGEST
From sensors & machines
CLEAN &
TRANSFORM
CPU Servers
EXPLORE
GPU Server
TRAIN
GPU Production Cluster
Access Pattern sequential sequential or random random random
Access Type write read & write read read
File Size mostly large small to large small to large mostly small
Concurrency high high low high

1.AIisaDataPipeline
2.Don’tThrowYourDataintoDataLake

DATA LAKE
OR DATA GRAVEYARD?
We see customers creating big data
graveyards, dumping everything into
HDFS [Hadoop Distributed File
System] and hoping to do something
with it down the road. But then they
just lose track of what’s there.
The main challenge is not creating a
data lake, but taking advantage of the
opportunities it presents.
“
”
PricewaterhouseCoopers
Technology Forecast, Issue 1, 2014

MODERN ANALYTICS WITH OLD DATA LAKE
SPRAWLING, COMPLEX SILOS & SLOW PERFORMANCE
Each App Locked into Physical Silos
Redundant Data Copies in Silos
Fixed Compute to Storage in Silo
Built for Large, Sequential Data
Optimized for Batch, Not Real-Time
STATIC DATA LAKE
NO LONGER VIABLE
HDFS DATA LAKE
SILO
SILO
SILOSILOSILO

1.AIisaDataPipeline
3.CloudorNottoCloud?

IT DEPENDS
WHERE YOU ARE ON YOUR AI JOURNEY
EXPLORATION PRODUCTION
NEED Start Immediately
Get New Products & Features to
Market Faster than Competition

IT DEPENDS
DON’T NEED Bogged Down with Infrastructure
Bogged Down by Performance
& Cost Inefficiencies

IT DEPENDS
DON’T NEED Bogged Down with Infrastructure
Bogged Down by Performance
& Cost Inefficiencies
RECOMMENDATION Cloud On-Premises

1.AIisaDataPipeline
4.Lies,DamnLies,andBenchmarks

BENCHMARKS DO NOT REFLECT REALITY
IMAGENET
REAL-WORLD AUTONOMOUS CAR
COMPANY
IMAGE SIZE 100-200KB 2-5MB
FILE SIZE 150MB
(Packed TFRecords)
2-5MB
MODE OF TESTING Synthetic (No I/O) Read from Storage

AI TRAINING SYSTEM
GOAL IS TO KEEP THE GPUs 100% BUSY
decode scale
evaluate
forward-
propagation
update
back-propagation
GPUI/O CPU
FULL TRAINING
WORKFLOW
Setup #1: Synthetic Data
from System RAM into
GPUs
Setup #3: Real Image Data from FlashBlade
into DGX-1
BENCHMARK
SETUP
GPU ONLY I/O + CPU + GPU
Setup #2: Real Image Data
from System RAM Through
CPU + GPU
CPU + GPU

NEAR-LINEAR SCALE DELIVERED
AIRI ENGINEERED FOR MAXIMUM PRODUCTIVITY AND OUT-OF-THE-BOX SCALE
DEEP LEARNING TRAINING- MULTI-NODE USING GPUDIRECT RDMA OVER ETHERNET
Comparing Synthetic Mode, Entire Data in DRAM, Entire Data in FlashBlade

1.AIisaDataPipeline
5.IdealDataPlatformisaDataHub

IDEAL PLATFORM FOR MODERN ERA
DYNAMIC DATA HUB ARCHITECTED FOR REAL-TIME & ELASTIC DATA
DATA PIPELINE
DATA HUB
“TUNED FOR EVERYTHING”
Small, Random to Large, Seq.
Architected for the Unknown
REAL-TIME
Low Latency Performance for
Instant Response
ALL-FLASH
Modern, Ultra-Fast
Technology
PARALLEL
No Serial Bottlenecks
for Max Throughput
ELASTIC
Grow Non-Disruptively
with More App Clusters
SIMPLE
Focus More on Insights,
Not Infrastructure

© 2018 PURE STORAGE INC. PURE PROPRIETARY26
NVIDIA® DGX-1™ | 4x DGX-1 Systems | 4 PFLOPS of DL Performance
PURE FLASHBLADE™ | 15x 17TB Blades | 1.5M IOPS
ARISTA | 2x 100Gb Ethernet Switches with RDMA
NVIDIA GPU CLOUD DEEP LEARNING STACK | NVIDIA Optimized Frameworks
AIRI SCALING TOOLKIT | Multi-node Training Made Simple
THE INDUSTRY’S FIRST
COMPLETE AI-READY INFRASTRUCTURE
HARDWARE
SOFTWARE

AI & MODERN ANALYTICS
POWERING ANALYTICS FOR WORLD’S LARGEST PUBLIC HEDGE FUND
AI CLEAN & LABEL AI EXPLORE AI TRAIN
CPU Servers GPU Server GPU Servers
SPARK
CPU Servers CPU Servers
MONGO
Our quants want to test a model,
get the results, and then test
another one- all day long. So a
10-20X improvement in
performance is a game-changer
when it comes to creating a
time-to-market advantage for us.
Gary Collier, co-CTO, Man AHL
“
”

ORCHESTRATION WITH OPENSHIFT
(KUBERNETES)
Monitoring
Load balancing
Scheduling
Resource allocation
OPENSHIFT + PURE PROVIDE RECIPE
FOR OPERATIONS AT SCALE

1.AIisaDataPipeline
5.IdealDataPlatformisaDataHub

Big Data LDN 2018: LESSONS LEARNED FROM DEPLOYING REAL-WORLD AI SYSTEMS

Big Data LDN 2018: LESSONS LEARNED FROM DEPLOYING REAL-WORLD AI SYSTEMS

Recommended

More Related Content

What's hot (20)

Similar to Big Data LDN 2018: LESSONS LEARNED FROM DEPLOYING REAL-WORLD AI SYSTEMS (20)

More from Matt Stubbs (20)

Recently uploaded (20)

Big Data LDN 2018: LESSONS LEARNED FROM DEPLOYING REAL-WORLD AI SYSTEMS