EXTENT-2016: Test Automation and Agile Testing

Nan Zhang
Senior QA Manager
GFT Technologies
Test Automation
and Agile Testing

Nan Zhang
June 2016
EXTENT 2016 CONFERENCE
Test Automation and
Agile Testing

GFT Group 27/06/2016 3
Executive Summary
GFT GROUP
 This presentation discusses the important role test automation plays in shaping Quality Assurance in Agile delivery
methodology and process. The topics include:
 Test Automation
 Test automation approach and benefits
 Test automation framework
 Industrialised test automation
 Agile Testing
 Agile delivery approach overview, activities, roles and responsibilities
 Agile testing scope, approach, and activities
 Agile testing phases and process
 Role of test automation in shaping Agile testing

GFT Group 27/06/2016 4
Test Automation Overview– Approach
GFT GROUP
 Approach
 Test automation solutions can be implemented through either custom-built solution or 3rd party products
 Test automation framework should be portable, reusable, easily maintained and designed to be transitioned to client’s team for
both GUI and server side automations
 Test automation solutions allow financial institutions to significantly increase test effectiveness and test coverage, therefore
ensuring a considerable reduction in “time to test” and decreased cost of testing over time
 It should be recognised that test automation comes at a cost and the predicted return on that investment should be achievable.
For that reason, the early estimation of effort and expected benefits is critical as is the ongoing collection of metrics to validate
that ROI
 The most practical automation solution should be pursued – whether using a known testing tool (e.g. QTP or Selenium, etc) or
through the development of own ‘fit for purpose’ test harness
 The approach is to provide deliverables (automated test suites) early and then on a regular basis such that the overall test
automation framework provides visible and increasing benefit – end-to-end
 Test automated solutions will, wherever feasible, be integrated into the build process to deliver an effective continuous
integration solution

GFT Group 27/06/2016 5
Test Automation Overview - Benefits
GFT GROUP
 Benefits
 Increased efficiency and coverage of tests through effective test automation
 Reduced overhead of regression testing
 Reduced cost of testing over time
 Reliability and predictability of tests by eliminating risk of human error
 Greater number of test runs possible in regression testing but also as smoke tests or within a continuous build /
integration environment
 Enables the realisation of Agile development methodology and delivers shippable product after each sprint

GFT Group 27/06/2016 6
 What is a “Test Automation Framework”?
 The key to successful test automation is to build a
solution that is usable by all test analysts, is robust
(can handle unexpected events and system errors),
reuses common test steps and is easily maintainable.
 Using a test automation tool ‘out of the box’ will only
cover these criteria to a small extent as the principle
behind most tools is to record the steps taken –
including all keystrokes, mouse operations, screen
navigation – that a manual tester performs to execute
a test such that the test can be played back multiple
times. This has numerous drawbacks: e.g. every
single test has to be recorded and so no reuse; as
soon as the system changes so all the tests have to
be re-recorded.
 The answer therefore is to build libraries of common
functions; parameterise the test scripts and externally
hold as much as possible in relation to the test
actions (using keywords) and test data. In addition
build error handling, smart reporting, test
dependencies and importantly an interface to the
underlying test automation tool that can be used by
all test analysts. These are all components of what is
referred to as the Test Automation Framework.
 The full framework encapsulates the automated regression test capability at
both the front (GUI) end and the server side (database/interfaces/messages);
the ability to manage the testing and the use of the framework; the tracking of
progress through dashboards and reports
Test Automation
Framework
Automated
RegressionScripts
Automated
Smoke Test
Scripts
Standards
Function
Libraries
Environment
Variables
Common
Libraries
Object
Repository
Error Handling
Tools and
Utilities
TestAutomationTool
(e.g.QTP,Selenium)
Data
Test ManagementArchive
Risk
Management
Trade
Processing
Finance &
Accounting
Pricing &
Analytics
Reporting &
Audit
Settlement
UpstreamSystems
DownstreamSystems
Application Under Test
Auto Test
GUI
 Usability
 Portability Maintainability
 Flexibility
 Robustness
 Reuse
GFT GROUP
The GFT Test Automation Framework

GFT Group 27/06/2016 7
 Description
 Industrialised test automation approach harmonises different
testing activities applied to the different subsystems in a large
scale systems integration project. This harmonization enables
reuse of testware, test environments, test simulations, test
automation and testing concepts for the systems integration
project.
 Approach
 Reach a testing maturity level where the test process is well
established and the corresponding working products are
defined and controlled, at which point reuse can take place
systematically.
 For synchronisation of the test processes between different
parties, an independent third party can be utilised which
guarantees that each involved party is working on reaching the
testing maturity level in sync with all other parties.
 Objective
 Overall goal is to increase test efficiency by a high degree of
reuse and to increase effectiveness by combining different test
environments to have more intermediate test stages on the
integration path.
 Benefits
 Many synergies by reusing testware that can be leveraged,
reducing test efforts significantly for all involved parties.
 Improve the overall quality of the system, since each involved
party can focus its test activities on those aspects that are core
for their deliverables
 Industrialised test automation enables layered automated
integration testing.
Industrialised Test Automation
GFT GROUP
 Industrialised test automation approach applies to a large scale systems integration project:

GFT Group 27/06/2016 8
Agile Delivery Approach - Overview
Project Kick Off
 Ensure that there are clearly defined
sprint and release goals (i.e. through to
delivery milestone).
 Agree and document the definition of
(‘Done’) at Sprint level and Release level
 Agree and publish the team’s sprint cycle
showing the length and schedule for
each sprint iteration.
 Define and document the triage
processes for story readiness and
defects
 Plan for automated testing and
continuous integration
Ongoing
 Keep the daily stand up brief: what I did
yesterday, what I will be doing today, any
blockers; take issues offline.
 The agreed Definition of ‘Done’ must be
adhered to
 Do not change the agreed sprint items unless
directed to by the Product Owner.
 Deploy each sprint to UAT or production
(depending on definition of ‘Done’)
2 31 4 5 6 7 8 9 10
ACTIVITIES
Deliver & Test Stories
Business & Regression Test
Code Freeze
Sprint Planning
Show & Tell
Retrospective
Daily Stand Up
Defect Triage
REGULAR
MEETINGS
Project
Working Group
ARTE-
FACTS
Delta Report
Commitment
List Project Status
Report
Project Status
Report
Backlog Backlog Refinement
Estimate &
Prioritise Stories
DAY
Deliver & Test Stories
Code Freeze & Deployment
Check
Point
Project
Working Group
Scrum of
Scrums
Show & Tell Rehearsal
GFT GROUP

GFT Group 27/06/2016 9
 The role of testing within an Agile development environment is of paramount importance with the testers fully integrated within the team so that testing
requirements can drive development
Agile Testing Approach
 The GFT ‘sprint testing’ approach includes:
 Test Driven Development (TDD)
 Behaviour Driven Development (BDD)
 Exploratory Testing
 Incremental Testing
Test Driven Development
Prioritised
requirements
backlog
Prioritised Manual and
Automated Test
Scripts
Daily Meetings
Daily Reports
(Metrics)
Continuous Automated
Regression Test
Release ready to
go live
Go-Live
Readiness
Weekly Test Reports
GFT Sprint Testing Approach
GFT GROUP
 Automation
 The implementation of a test
automation solution is essential and
will bring significant benefits in an
Agile environment.
 Automating tests as early as possible
and building a regression pack to run
on a daily basis

GFT Group 27/06/2016 10
 GFT implementation teams focus on the following areas in the context of Agile testing.
GFT GROUP
Agile Testing Scope and Activities
 Designing a testing framework and infrastructure can be as complex as designing and implementing
the system being tested. Test analysts focus on doing the technical design of testing frameworks that
are able to meet requirements for testing individual releases and ongoing system test and certification
 Functional test design requires tight alignment between the business analysts that are defining the
functional capability of the system and the test analysts that are defining the test cases. Test analysts
and engineers should be deployed to work as part of an integrated design and development team and
ensure the specification of comprehensive test coverage
 Test automation experts engage with clients to design and implement test automation frameworks that
are portable, reusable, maintainable, and able to achieve increased test effectiveness, reduced testing
time and reduced overall cost. Test automation tools are analysed and selected for both functional and
non-functional test.
 One of the most difficult aspects of testing is creating and managing the data that drives the testing
framework. Test teams deploy for clients a project specific approach to the management and masking
of real production data and the generation of comprehensive test data where the use of live data is not
possible
 The test team is able to provide strategic direction and hands-on implementations around non-
functional testing and in particular, performance tuning, performance testing, system availability testing
and security
Test Design
Functional Testing
Test Automation
Test Data Management
Non-Functional
Testing

GFT Group 27/06/2016 11
 Testing for individual user stories are done
within the sprint (i.e. the definition of Done
includes the completion of testing)
 A continuous integration environment is in
place to enable code deployed daily and
also on-demand.
 Automated test suites kicked off after code
deployment
 SIT and UAT are run after each sprint in
an iterative and progressive manner
 Each sprint aims to deliver a shippable
product to production environment
 Test automation and Continuous
Integration plays vital role in the realisation
of such approach
GFT GROUP
Agile Testing Phases and Process
SprintNstart
Sprint N (Sprint test)
SprintNend
Sprint N deployed
to SIT env
UAT execution
ProdDeployable
Continuous Integration Test
CIDeployment
SprintN+1start
SprintN+1end
Product Backlog Refinement
Sprint N deployed
to UAT env
Activity
Test Phase

GFT Group 27/06/2016 12
Test Automation in Agile Testing
GFT GROUP
Levels and areas
 Unit tests
 Backend (DB, services)
 UI (validating behaviour or UX design)
 Continuous Integration Testing (daily
build and automated test)
 Full E2E solutions (with all system
components and dependency systems)
 Non-functional testing (Performance
validation, soak test)
Implementation
 Test framework based on open source, custom solution or mix
 Building re-usable solutions (reducing next project kick-off cost)
 Detailed test cases prepared by QA
 Tests can be written by QA or Dev (fast test coverage increase if
needed)
 Coverage can be measured by traceability matrix
 All tests integrated with CI environment to produce daily reports
 Test automation built for both functional and non-functional test

EXTENT-2016: Test Automation and Agile Testing

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to EXTENT-2016: Test Automation and Agile Testing (20)

More from Iosif Itkin (20)

Recently uploaded (20)

EXTENT-2016: Test Automation and Agile Testing