SlideShare a Scribd company logo
1Webinar
Presents
with
Agile Load Testing in the
Real World
2
Continuous Load Testing Realized
TODAY’S PRESENTERS
Lee Barnes: Utopia Solutions, Founder & CTO
Mike Ostenberg: SOASTA, Dir. Systems Engineering
Brad Johnson: SOASTA, VP Product
Agenda
• A case study in driving a performance culture with agile delivery
• A platform for delivering continuous load testing
Questions:
Submit in chat box during event
3
© Utopia Solutions
Agile Load Testing in the Real World
Lee Barnes, CTO
Utopia Solutions
@USI_LeeBarnes
© Utopia Solutions5
Key Questions
What needs to change?
How do we make those changes?
Why don’t traditional performance
testing models fit nicely with agile?
© Utopia Solutions6
Case Study
Background
• Large global manufacturer
• Performance Testing CoE in operation for ~10 yrs
• Seen as a huge value to the enterprise
• Began moving toward agile a few years ago…
and the luster of the CoE began to dull…
© Utopia Solutions7
Traditional Performance Testing Model
Provides Consistency in…
•Resources / Skills
•Processes
•Deliverable Quality
•Technology
Request Test
Project
Team
Center of
Excellence
Perform Discovery
Develop & Execute
Deliver ResultsEvaluate Results
Info / Environment
© Utopia Solutions8
• One size fits all
• Strict entry criteria
• Performed late in SDLC
• One-way communication of
results
Service Characteristics
CoE in the Waterfall Years
Performance Testing
Too late!
© Utopia Solutions9
Agile Development
Deploy
Where does performance
testing fit?
© Utopia Solutions10
Early Attempts on Agile Projects
Agile
Performance Testing
Same Process…
Same Risk!
© Utopia Solutions11
Early Attempts on Agile Projects
Agile
Performance
Testing
Performance
Testing
Performance
Testing
Performance
Testing
CoE Approach
•Rigid
•4 week test cycle
Agile Approach
•Flexible
•2 week sprints
+ =
© Utopia Solutions
Back to the drawing
board…
© Utopia Solutions13
Early Cycle /
Ongoing
Performance
Analysis
Establish Objectives
Ultimate Goal…
Late Cycle
Performance
Testing
Shift Left
© Utopia Solutions14
Identify Gaps
What’s stopping us from analyzing performance
continuously?
• Performance objectives weren’t part of
user stories
• Agile teams lacked performance
engineering expertise
• Performance testing knowledge and tools
were locked in the CoE
• APM tools / skills were non-existent
© Utopia Solutions15
Disclaimers
It was not easy.
The changes discussed on the next few
slides were implemented over a two
year duration.
It was not painless.
© Utopia Solutions16
Performance Testing in an Agile Environment
Build performance objectives in to user
stories
Make performance everyone’s job
Create a role to coordinate
performance activities
CoE performance test architects
become mentors
Begin developing a
performance focused
culture
© Utopia Solutions17
Performance Testing in an Agile Environment
Map business and end user
requirements onto system components
Create both single and multi user tests
Benchmark in development / QA
environment
Look for trends in measurements vs.
absolutes
Use service virtualization to stub out 3rd
party or yet to be developed
components
Execute unit performance tests as part
of continuous integration
Implement Unit /
Component
Performance Testing
© Utopia Solutions18
Waterfall Agile
Responsibility for
Performance
Performance CoE Everyone
Role of CoE End of cycle PT Provide mentorship to agile team
Pre-deployment system PT
Role of
development
Throw system over wall
Wait for results
Continuous focus on performance
Tools Traditional load test tools Traditional load test tools
APM tools
Service virtualization
Continuous integration
Environment Dedicated / production-like Development / desktop
Deliverable Boiler plate results delivered
just prior to release
Continuous feed of actionable
data
Performance Testing in an Agile Environment
Transformation Summary
Waterfall Agile
Responsibility for
Performance
Performance CoE
Role of CoE End of cycle PT
Role of
development
Throw system over wall
Wait for results
Tools Traditional load test tools
Environment Dedicated / production-like
Deliverable Boiler plate results delivered
just prior to release
© Utopia Solutions19
Summary
Key takeaways…
•Agile performance testing is not more /
quicker performance tests
•Success requires an shift in people,
process and technology
20
Mike Ostenberg
21
Direct Questions?
Lee Barnes
Founder and CTO
Email:lee.barnes@utopiasolutions.com
twitter.com/USI_LeeBarnes
linkedin.com/in/leebarnes
Blog
www.RaiseYourMQ.com
Thanks & Q&A
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twitter.com/cloudtesttwitter.com/cloudtest
facebook.com/cloudtestfacebook.com/cloudtest
Knowledge Center
•White Papers
•Webinar
Recordings
•Case Studies
CloudLink
Community
•Support
•Tutorials
•Video
RESOURCES
Contact Utopia
www.utopiasolutions.com
630.375.6610
www.SOASTA.com www.UtopiaSolutions.com
22
www.SOASTA.com

More Related Content

What's hot (20)

Introducing DevOps, IT Sharing Session 20 Nov 2017
Introducing DevOps, IT Sharing Session 20 Nov 2017Introducing DevOps, IT Sharing Session 20 Nov 2017
Introducing DevOps, IT Sharing Session 20 Nov 2017
Danny Ariwicaksono
 
Agile EVM
Agile EVMAgile EVM
Agile EVM
Nishanth K Hydru
 
Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.
Worksoft
 
10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA
Jim Spillson
 
Continuous testing for devops
Continuous testing for devopsContinuous testing for devops
Continuous testing for devops
Subrahmaniam S.R.V
 
A paradigm shift for testing - how to increase productivity 10x!
A paradigm shift for testing - how to increase productivity 10x!A paradigm shift for testing - how to increase productivity 10x!
A paradigm shift for testing - how to increase productivity 10x!
Vasco Duarte
 
Dev ops != Dev+Ops
Dev ops != Dev+OpsDev ops != Dev+Ops
Dev ops != Dev+Ops
Shalu Ahuja
 
A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
XebiaLabs
 
Building DevOps Toolchain
Building DevOps ToolchainBuilding DevOps Toolchain
Building DevOps Toolchain
IBM UrbanCode Products
 
Reinventing Performance Testing, CMG imPACt 2016 slides
  Reinventing Performance Testing, CMG imPACt 2016 slides  Reinventing Performance Testing, CMG imPACt 2016 slides
Reinventing Performance Testing, CMG imPACt 2016 slides
Alexander Podelko
 
Branch Management and Atomic Merge in a Continuous Integration Environment
Branch Management and Atomic Merge in a Continuous Integration Environment  Branch Management and Atomic Merge in a Continuous Integration Environment
Branch Management and Atomic Merge in a Continuous Integration Environment
Perforce
 
What's the State of Agile Software Development?
What's the State of Agile Software Development?What's the State of Agile Software Development?
What's the State of Agile Software Development?
VersionOne
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
Perfecto Mobile
 
Shifting Left Using Sauce Analytics
Shifting Left Using Sauce AnalyticsShifting Left Using Sauce Analytics
Shifting Left Using Sauce Analytics
Sauce Labs
 
Continuous Performance Testing: Myths and Realities
Continuous Performance Testing: Myths and RealitiesContinuous Performance Testing: Myths and Realities
Continuous Performance Testing: Myths and Realities
Alexander Podelko
 
Continuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOpsContinuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOps
SeaLights
 
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
VersionOne
 
Agile Test Automation
Agile Test AutomationAgile Test Automation
Agile Test Automation
Werner Keil
 
Continuous Testing: A Key to DevOps Success
Continuous Testing: A Key to DevOps SuccessContinuous Testing: A Key to DevOps Success
Continuous Testing: A Key to DevOps Success
TechWell
 
DevOps presentation
DevOps presentationDevOps presentation
DevOps presentation
Axsh Co. LTD
 
Introducing DevOps, IT Sharing Session 20 Nov 2017
Introducing DevOps, IT Sharing Session 20 Nov 2017Introducing DevOps, IT Sharing Session 20 Nov 2017
Introducing DevOps, IT Sharing Session 20 Nov 2017
Danny Ariwicaksono
 
Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.
Worksoft
 
10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA
Jim Spillson
 
A paradigm shift for testing - how to increase productivity 10x!
A paradigm shift for testing - how to increase productivity 10x!A paradigm shift for testing - how to increase productivity 10x!
A paradigm shift for testing - how to increase productivity 10x!
Vasco Duarte
 
Dev ops != Dev+Ops
Dev ops != Dev+OpsDev ops != Dev+Ops
Dev ops != Dev+Ops
Shalu Ahuja
 
A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
XebiaLabs
 
Reinventing Performance Testing, CMG imPACt 2016 slides
  Reinventing Performance Testing, CMG imPACt 2016 slides  Reinventing Performance Testing, CMG imPACt 2016 slides
Reinventing Performance Testing, CMG imPACt 2016 slides
Alexander Podelko
 
Branch Management and Atomic Merge in a Continuous Integration Environment
Branch Management and Atomic Merge in a Continuous Integration Environment  Branch Management and Atomic Merge in a Continuous Integration Environment
Branch Management and Atomic Merge in a Continuous Integration Environment
Perforce
 
What's the State of Agile Software Development?
What's the State of Agile Software Development?What's the State of Agile Software Development?
What's the State of Agile Software Development?
VersionOne
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
Perfecto Mobile
 
Shifting Left Using Sauce Analytics
Shifting Left Using Sauce AnalyticsShifting Left Using Sauce Analytics
Shifting Left Using Sauce Analytics
Sauce Labs
 
Continuous Performance Testing: Myths and Realities
Continuous Performance Testing: Myths and RealitiesContinuous Performance Testing: Myths and Realities
Continuous Performance Testing: Myths and Realities
Alexander Podelko
 
Continuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOpsContinuous Testing - What QA means for DevOps
Continuous Testing - What QA means for DevOps
SeaLights
 
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
AgileLIVE: Scaling Agile Faster, Easier, Smarter with SAFe and VersionOne - P...
VersionOne
 
Agile Test Automation
Agile Test AutomationAgile Test Automation
Agile Test Automation
Werner Keil
 
Continuous Testing: A Key to DevOps Success
Continuous Testing: A Key to DevOps SuccessContinuous Testing: A Key to DevOps Success
Continuous Testing: A Key to DevOps Success
TechWell
 
DevOps presentation
DevOps presentationDevOps presentation
DevOps presentation
Axsh Co. LTD
 

Similar to Agile Load Testing In The Real World (20)

Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
TechWell
 
Test automation lessons from WebSphere Application Server
Test automation lessons from WebSphere Application ServerTest automation lessons from WebSphere Application Server
Test automation lessons from WebSphere Application Server
Robbie Minshall
 
A True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOpsA True Story of Why QA Loves DevOps
A True Story of Why QA Loves DevOps
IBM UrbanCode Products
 
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Emerasoft, solutions to collaborate
 
Kristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's SeatKristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's Seat
TEST Huddle
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
Andrew Williams
 
Soasta | CloudBees webinar 11/12/2015
Soasta | CloudBees webinar 11/12/2015Soasta | CloudBees webinar 11/12/2015
Soasta | CloudBees webinar 11/12/2015
SOASTA
 
Automated Testing Tutorial
Automated Testing TutorialAutomated Testing Tutorial
Automated Testing Tutorial
John Liebenau
 
Agile performance testing
Agile performance testingAgile performance testing
Agile performance testing
Andriy Melnyk
 
Achal_Resume_7.4
Achal_Resume_7.4Achal_Resume_7.4
Achal_Resume_7.4
Achal Dalvi
 
Agile testing
Agile testingAgile testing
Agile testing
Raj Indugula
 
Building a Test Automation Strategy for Success
Building a Test Automation Strategy for SuccessBuilding a Test Automation Strategy for Success
Building a Test Automation Strategy for Success
Lee Barnes
 
Continuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee BarnesContinuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee Barnes
QA or the Highway
 
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
Aberla
 
6.11.2013 - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
6.11.2013   - 2013 - Continuous Delivery - Testing for Agile Through Continuo...6.11.2013   - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
6.11.2013 - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
IBM Rational
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation
Worksoft
 
CEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.pptCEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.ppt
nabeenbeen
 
CEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.pptCEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.ppt
fsfsdffs
 
CEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.pptCEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.ppt
AnkurGarg165647
 
Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
TechWell
 
Test automation lessons from WebSphere Application Server
Test automation lessons from WebSphere Application ServerTest automation lessons from WebSphere Application Server
Test automation lessons from WebSphere Application Server
Robbie Minshall
 
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Emerasoft, solutions to collaborate
 
Kristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's SeatKristian Fischer - Put Test in the Driver's Seat
Kristian Fischer - Put Test in the Driver's Seat
TEST Huddle
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
Andrew Williams
 
Soasta | CloudBees webinar 11/12/2015
Soasta | CloudBees webinar 11/12/2015Soasta | CloudBees webinar 11/12/2015
Soasta | CloudBees webinar 11/12/2015
SOASTA
 
Automated Testing Tutorial
Automated Testing TutorialAutomated Testing Tutorial
Automated Testing Tutorial
John Liebenau
 
Agile performance testing
Agile performance testingAgile performance testing
Agile performance testing
Andriy Melnyk
 
Achal_Resume_7.4
Achal_Resume_7.4Achal_Resume_7.4
Achal_Resume_7.4
Achal Dalvi
 
Building a Test Automation Strategy for Success
Building a Test Automation Strategy for SuccessBuilding a Test Automation Strategy for Success
Building a Test Automation Strategy for Success
Lee Barnes
 
Continuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee BarnesContinuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee Barnes
QA or the Highway
 
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
ESEconf2011 - Guckenheimer Sam: "Agile in the Very Large"
Aberla
 
6.11.2013 - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
6.11.2013   - 2013 - Continuous Delivery - Testing for Agile Through Continuo...6.11.2013   - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
6.11.2013 - 2013 - Continuous Delivery - Testing for Agile Through Continuo...
IBM Rational
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation
Worksoft
 
CEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.pptCEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.ppt
nabeenbeen
 
CEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.pptCEN6070-Chapter7.1.ppt
CEN6070-Chapter7.1.ppt
fsfsdffs
 

More from SOASTA (20)

Mobile Performance: State of the Union
Mobile Performance: State of the UnionMobile Performance: State of the Union
Mobile Performance: State of the Union
SOASTA
 
Techniques, Tips & Tools For Mobile App Testing
Techniques, Tips & Tools For Mobile App TestingTechniques, Tips & Tools For Mobile App Testing
Techniques, Tips & Tools For Mobile App Testing
SOASTA
 
Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)
SOASTA
 
Metrics, Metrics Everywhere (but where the heck do you start?)
Metrics, Metrics Everywhere (but where the heck do you start?)Metrics, Metrics Everywhere (but where the heck do you start?)
Metrics, Metrics Everywhere (but where the heck do you start?)
SOASTA
 
3 tips to increase mobile test coverage
3 tips to increase mobile test coverage3 tips to increase mobile test coverage
3 tips to increase mobile test coverage
SOASTA
 
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & MobileWebinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
SOASTA
 
Get Ready for Changes To Load Testing
Get Ready for Changes To Load Testing Get Ready for Changes To Load Testing
Get Ready for Changes To Load Testing
SOASTA
 
Building a Performance A-Team
Building a Performance A-TeamBuilding a Performance A-Team
Building a Performance A-Team
SOASTA
 
Dach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
Dach webinar - Image Absicherung – Lektionen aus dem Facebook CrashDach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
Dach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
SOASTA
 
Four best practices for performance testing mobile apps soasta and utopia
Four best practices for performance testing mobile apps   soasta and utopiaFour best practices for performance testing mobile apps   soasta and utopia
Four best practices for performance testing mobile apps soasta and utopia
SOASTA
 
Synthetic and rum webinar
Synthetic and rum webinarSynthetic and rum webinar
Synthetic and rum webinar
SOASTA
 
7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing
SOASTA
 
Secrets to Realistic Load Testing
Secrets to Realistic Load TestingSecrets to Realistic Load Testing
Secrets to Realistic Load Testing
SOASTA
 
How to measure the business impact of web performance
How to measure the business impact of web performanceHow to measure the business impact of web performance
How to measure the business impact of web performance
SOASTA
 
Lasttest auf Zuruf CloudTest on Demand webinar presentation
Lasttest auf Zuruf CloudTest on Demand webinar presentationLasttest auf Zuruf CloudTest on Demand webinar presentation
Lasttest auf Zuruf CloudTest on Demand webinar presentation
SOASTA
 
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
Accelerate Web and Mobile Testing for Continuous Integration and DeliveryAccelerate Web and Mobile Testing for Continuous Integration and Delivery
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
SOASTA
 
The Mobile Testing Checklist
The Mobile Testing ChecklistThe Mobile Testing Checklist
The Mobile Testing Checklist
SOASTA
 
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
Reducing 3rd party content risk with Real User Monitoring
Reducing 3rd party content risk with Real User MonitoringReducing 3rd party content risk with Real User Monitoring
Reducing 3rd party content risk with Real User Monitoring
SOASTA
 
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal ReadinessTis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
SOASTA
 
Mobile Performance: State of the Union
Mobile Performance: State of the UnionMobile Performance: State of the Union
Mobile Performance: State of the Union
SOASTA
 
Techniques, Tips & Tools For Mobile App Testing
Techniques, Tips & Tools For Mobile App TestingTechniques, Tips & Tools For Mobile App Testing
Techniques, Tips & Tools For Mobile App Testing
SOASTA
 
Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)
SOASTA
 
Metrics, Metrics Everywhere (but where the heck do you start?)
Metrics, Metrics Everywhere (but where the heck do you start?)Metrics, Metrics Everywhere (but where the heck do you start?)
Metrics, Metrics Everywhere (but where the heck do you start?)
SOASTA
 
3 tips to increase mobile test coverage
3 tips to increase mobile test coverage3 tips to increase mobile test coverage
3 tips to increase mobile test coverage
SOASTA
 
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & MobileWebinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
Webinar: Was die Top eCommerce Firmen über Ihre Performance auf Web- & Mobile
SOASTA
 
Get Ready for Changes To Load Testing
Get Ready for Changes To Load Testing Get Ready for Changes To Load Testing
Get Ready for Changes To Load Testing
SOASTA
 
Building a Performance A-Team
Building a Performance A-TeamBuilding a Performance A-Team
Building a Performance A-Team
SOASTA
 
Dach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
Dach webinar - Image Absicherung – Lektionen aus dem Facebook CrashDach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
Dach webinar - Image Absicherung – Lektionen aus dem Facebook Crash
SOASTA
 
Four best practices for performance testing mobile apps soasta and utopia
Four best practices for performance testing mobile apps   soasta and utopiaFour best practices for performance testing mobile apps   soasta and utopia
Four best practices for performance testing mobile apps soasta and utopia
SOASTA
 
Synthetic and rum webinar
Synthetic and rum webinarSynthetic and rum webinar
Synthetic and rum webinar
SOASTA
 
7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing
SOASTA
 
Secrets to Realistic Load Testing
Secrets to Realistic Load TestingSecrets to Realistic Load Testing
Secrets to Realistic Load Testing
SOASTA
 
How to measure the business impact of web performance
How to measure the business impact of web performanceHow to measure the business impact of web performance
How to measure the business impact of web performance
SOASTA
 
Lasttest auf Zuruf CloudTest on Demand webinar presentation
Lasttest auf Zuruf CloudTest on Demand webinar presentationLasttest auf Zuruf CloudTest on Demand webinar presentation
Lasttest auf Zuruf CloudTest on Demand webinar presentation
SOASTA
 
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
Accelerate Web and Mobile Testing for Continuous Integration and DeliveryAccelerate Web and Mobile Testing for Continuous Integration and Delivery
Accelerate Web and Mobile Testing for Continuous Integration and Delivery
SOASTA
 
The Mobile Testing Checklist
The Mobile Testing ChecklistThe Mobile Testing Checklist
The Mobile Testing Checklist
SOASTA
 
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
How To Use Jenkins for Continuous Load and Mobile Testing with SOASTA & Cloud...
SOASTA
 
Reducing 3rd party content risk with Real User Monitoring
Reducing 3rd party content risk with Real User MonitoringReducing 3rd party content risk with Real User Monitoring
Reducing 3rd party content risk with Real User Monitoring
SOASTA
 
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal ReadinessTis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
Tis The Season: Load Testing Tips and Checklist for Retail Seasonal Readiness
SOASTA
 

Recently uploaded (20)

Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptxIn-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
aptyai
 
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Alan Dix
 
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Maarten Verwaest
 
Config 2025 presentation recap covering both days
Config 2025 presentation recap covering both daysConfig 2025 presentation recap covering both days
Config 2025 presentation recap covering both days
TrishAntoni1
 
Mastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B LandscapeMastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B Landscape
marketing943205
 
AI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamsonAI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamson
UXPA Boston
 
React Native for Business Solutions: Building Scalable Apps for Success
React Native for Business Solutions: Building Scalable Apps for SuccessReact Native for Business Solutions: Building Scalable Apps for Success
React Native for Business Solutions: Building Scalable Apps for Success
Amelia Swank
 
May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Building the Customer Identity Community, Together.pdf
Building the Customer Identity Community, Together.pdfBuilding the Customer Identity Community, Together.pdf
Building the Customer Identity Community, Together.pdf
Cheryl Hung
 
Top 5 Qualities to Look for in Salesforce Partners in 2025
Top 5 Qualities to Look for in Salesforce Partners in 2025Top 5 Qualities to Look for in Salesforce Partners in 2025
Top 5 Qualities to Look for in Salesforce Partners in 2025
Damco Salesforce Services
 
DNF 2.0 Implementations Challenges in Nepal
DNF 2.0 Implementations Challenges in NepalDNF 2.0 Implementations Challenges in Nepal
DNF 2.0 Implementations Challenges in Nepal
ICT Frame Magazine Pvt. Ltd.
 
Cybersecurity Threat Vectors and Mitigation
Cybersecurity Threat Vectors and MitigationCybersecurity Threat Vectors and Mitigation
Cybersecurity Threat Vectors and Mitigation
VICTOR MAESTRE RAMIREZ
 
Dark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanizationDark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanization
Jakub Šimek
 
Building a research repository that works by Clare Cady
Building a research repository that works by Clare CadyBuilding a research repository that works by Clare Cady
Building a research repository that works by Clare Cady
UXPA Boston
 
Sustainable_Development_Goals_INDIANWraa
Sustainable_Development_Goals_INDIANWraaSustainable_Development_Goals_INDIANWraa
Sustainable_Development_Goals_INDIANWraa
03ANMOLCHAURASIYA
 
Understanding SEO in the Age of AI.pdf
Understanding SEO in the Age of AI.pdfUnderstanding SEO in the Age of AI.pdf
Understanding SEO in the Age of AI.pdf
Fulcrum Concepts, LLC
 
Design pattern talk by Kaya Weers - 2025 (v2)
Design pattern talk by Kaya Weers - 2025 (v2)Design pattern talk by Kaya Weers - 2025 (v2)
Design pattern talk by Kaya Weers - 2025 (v2)
Kaya Weers
 
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
Lorenzo Miniero
 
Artificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptxArtificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptx
03ANMOLCHAURASIYA
 
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Kit-Works Team Study_아직도 Dockefile.pdf_김성호
Wonjun Hwang
 
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptxIn-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
In-App Guidance_ Save Enterprises Millions in Training & IT Costs.pptx
aptyai
 
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Who's choice? Making decisions with and about Artificial Intelligence, Keele ...
Alan Dix
 
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Limecraft Webinar - 2025.3 release, featuring Content Delivery, Graphic Conte...
Maarten Verwaest
 
Config 2025 presentation recap covering both days
Config 2025 presentation recap covering both daysConfig 2025 presentation recap covering both days
Config 2025 presentation recap covering both days
TrishAntoni1
 
Mastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B LandscapeMastering Testing in the Modern F&B Landscape
Mastering Testing in the Modern F&B Landscape
marketing943205
 
AI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamsonAI-proof your career by Olivier Vroom and David WIlliamson
AI-proof your career by Olivier Vroom and David WIlliamson
UXPA Boston
 
React Native for Business Solutions: Building Scalable Apps for Success
React Native for Business Solutions: Building Scalable Apps for SuccessReact Native for Business Solutions: Building Scalable Apps for Success
React Native for Business Solutions: Building Scalable Apps for Success
Amelia Swank
 
May Patch Tuesday
May Patch TuesdayMay Patch Tuesday
May Patch Tuesday
Ivanti
 
Building the Customer Identity Community, Together.pdf
Building the Customer Identity Community, Together.pdfBuilding the Customer Identity Community, Together.pdf
Building the Customer Identity Community, Together.pdf
Cheryl Hung
 
Top 5 Qualities to Look for in Salesforce Partners in 2025
Top 5 Qualities to Look for in Salesforce Partners in 2025Top 5 Qualities to Look for in Salesforce Partners in 2025
Top 5 Qualities to Look for in Salesforce Partners in 2025
Damco Salesforce Services
 
Cybersecurity Threat Vectors and Mitigation
Cybersecurity Threat Vectors and MitigationCybersecurity Threat Vectors and Mitigation
Cybersecurity Threat Vectors and Mitigation
VICTOR MAESTRE RAMIREZ
 
Dark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanizationDark Dynamism: drones, dark factories and deurbanization
Dark Dynamism: drones, dark factories and deurbanization
Jakub Šimek
 
Building a research repository that works by Clare Cady
Building a research repository that works by Clare CadyBuilding a research repository that works by Clare Cady
Building a research repository that works by Clare Cady
UXPA Boston
 
Sustainable_Development_Goals_INDIANWraa
Sustainable_Development_Goals_INDIANWraaSustainable_Development_Goals_INDIANWraa
Sustainable_Development_Goals_INDIANWraa
03ANMOLCHAURASIYA
 
Understanding SEO in the Age of AI.pdf
Understanding SEO in the Age of AI.pdfUnderstanding SEO in the Age of AI.pdf
Understanding SEO in the Age of AI.pdf
Fulcrum Concepts, LLC
 
Design pattern talk by Kaya Weers - 2025 (v2)
Design pattern talk by Kaya Weers - 2025 (v2)Design pattern talk by Kaya Weers - 2025 (v2)
Design pattern talk by Kaya Weers - 2025 (v2)
Kaya Weers
 
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
Lorenzo Miniero
 
Artificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptxArtificial_Intelligence_in_Everyday_Life.pptx
Artificial_Intelligence_in_Everyday_Life.pptx
03ANMOLCHAURASIYA
 

Agile Load Testing In The Real World

  • 2. 2 Continuous Load Testing Realized TODAY’S PRESENTERS Lee Barnes: Utopia Solutions, Founder & CTO Mike Ostenberg: SOASTA, Dir. Systems Engineering Brad Johnson: SOASTA, VP Product Agenda • A case study in driving a performance culture with agile delivery • A platform for delivering continuous load testing Questions: Submit in chat box during event
  • 3. 3
  • 4. © Utopia Solutions Agile Load Testing in the Real World Lee Barnes, CTO Utopia Solutions @USI_LeeBarnes
  • 5. © Utopia Solutions5 Key Questions What needs to change? How do we make those changes? Why don’t traditional performance testing models fit nicely with agile?
  • 6. © Utopia Solutions6 Case Study Background • Large global manufacturer • Performance Testing CoE in operation for ~10 yrs • Seen as a huge value to the enterprise • Began moving toward agile a few years ago… and the luster of the CoE began to dull…
  • 7. © Utopia Solutions7 Traditional Performance Testing Model Provides Consistency in… •Resources / Skills •Processes •Deliverable Quality •Technology Request Test Project Team Center of Excellence Perform Discovery Develop & Execute Deliver ResultsEvaluate Results Info / Environment
  • 8. © Utopia Solutions8 • One size fits all • Strict entry criteria • Performed late in SDLC • One-way communication of results Service Characteristics CoE in the Waterfall Years Performance Testing Too late!
  • 9. © Utopia Solutions9 Agile Development Deploy Where does performance testing fit?
  • 10. © Utopia Solutions10 Early Attempts on Agile Projects Agile Performance Testing Same Process… Same Risk!
  • 11. © Utopia Solutions11 Early Attempts on Agile Projects Agile Performance Testing Performance Testing Performance Testing Performance Testing CoE Approach •Rigid •4 week test cycle Agile Approach •Flexible •2 week sprints + =
  • 12. © Utopia Solutions Back to the drawing board…
  • 13. © Utopia Solutions13 Early Cycle / Ongoing Performance Analysis Establish Objectives Ultimate Goal… Late Cycle Performance Testing Shift Left
  • 14. © Utopia Solutions14 Identify Gaps What’s stopping us from analyzing performance continuously? • Performance objectives weren’t part of user stories • Agile teams lacked performance engineering expertise • Performance testing knowledge and tools were locked in the CoE • APM tools / skills were non-existent
  • 15. © Utopia Solutions15 Disclaimers It was not easy. The changes discussed on the next few slides were implemented over a two year duration. It was not painless.
  • 16. © Utopia Solutions16 Performance Testing in an Agile Environment Build performance objectives in to user stories Make performance everyone’s job Create a role to coordinate performance activities CoE performance test architects become mentors Begin developing a performance focused culture
  • 17. © Utopia Solutions17 Performance Testing in an Agile Environment Map business and end user requirements onto system components Create both single and multi user tests Benchmark in development / QA environment Look for trends in measurements vs. absolutes Use service virtualization to stub out 3rd party or yet to be developed components Execute unit performance tests as part of continuous integration Implement Unit / Component Performance Testing
  • 18. © Utopia Solutions18 Waterfall Agile Responsibility for Performance Performance CoE Everyone Role of CoE End of cycle PT Provide mentorship to agile team Pre-deployment system PT Role of development Throw system over wall Wait for results Continuous focus on performance Tools Traditional load test tools Traditional load test tools APM tools Service virtualization Continuous integration Environment Dedicated / production-like Development / desktop Deliverable Boiler plate results delivered just prior to release Continuous feed of actionable data Performance Testing in an Agile Environment Transformation Summary Waterfall Agile Responsibility for Performance Performance CoE Role of CoE End of cycle PT Role of development Throw system over wall Wait for results Tools Traditional load test tools Environment Dedicated / production-like Deliverable Boiler plate results delivered just prior to release
  • 19. © Utopia Solutions19 Summary Key takeaways… •Agile performance testing is not more / quicker performance tests •Success requires an shift in people, process and technology
  • 21. 21 Direct Questions? Lee Barnes Founder and CTO Email:lee.barnes@utopiasolutions.com twitter.com/USI_LeeBarnes linkedin.com/in/leebarnes Blog www.RaiseYourMQ.com Thanks & Q&A Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: twitter.com/cloudtesttwitter.com/cloudtest facebook.com/cloudtestfacebook.com/cloudtest Knowledge Center •White Papers •Webinar Recordings •Case Studies CloudLink Community •Support •Tutorials •Video RESOURCES Contact Utopia www.utopiasolutions.com 630.375.6610 www.SOASTA.com www.UtopiaSolutions.com
  翻译: