Performance testing material

Software Performance TestingSoftware Performance Testing
Load Runner TrainingLoad Runner Training

Possible over-interaction between application servers and database servers
Sizing of packets (size in application versus size on network)
Connection pools and connection sharing
Location of servers on the network
Increased latency and jitter
Application issues:
Excessive memory allocation and de-allocation
Improper task initialization and housekeeping
Improper garbage collection, especially after a fault or failure
Loss of sessions or sessions kept alive too long (timers)
Persistent applications like BlackBerry devices
Application configuration parameter conflicts
High resource consumption features (CAD/CAM, etc.)
Database issues:
Index design
Use of dynamic indexes
Potential deadlocks from locking contention
Inefficient use of cache
Overuse of cache
Use of resource-intensive features such as referential integrity
Use of stored procedures or trigger events
Table fragmentation (overuse of third normal form)
Improper timing of table and index reorganization

• Performance Testing determines how fast a systemPerformance Testing determines how fast a system
performs under a particular workload to validate theperforms under a particular workload to validate the
quality attributes of the system such as scalability andquality attributes of the system such as scalability and
reliability and Resource usage of the Application Underreliability and Resource usage of the Application Under
Test.Test.
• It explores several system qualities, that can beIt explores several system qualities, that can be
simplified to:simplified to:
Speed - does the system respond quickly enough?Speed - does the system respond quickly enough?
Capacity - is the infrastructure sized satisfactorily ?Capacity - is the infrastructure sized satisfactorily ?
Scalability - can the system grow to handle futureScalability - can the system grow to handle future
volumes?volumes?
Stability - does the system behave correctly under load?Stability - does the system behave correctly under load?
WHAT IS PERFORMANCE
TESTING

Why Do Performance Testing?Why Do Performance Testing?
• Obtain an accurate picture of end-to-end system performanceObtain an accurate picture of end-to-end system performance
before going live.before going live.
• How The System is Capable of handling future growth.How The System is Capable of handling future growth.
• Whether Application requires a performance improvement/hardwareWhether Application requires a performance improvement/hardware
upgrade prior to release.upgrade prior to release.
• Revenue losses or damaged brand credibility due to scalability orRevenue losses or damaged brand credibility due to scalability or
stability issues.stability issues.
• Customers being dissatisfied with application response time.Customers being dissatisfied with application response time.
• Analyzing the behavior of the application at various load levels.Analyzing the behavior of the application at various load levels.
• Identifying bottlenecks in the application.Identifying bottlenecks in the application.

Why do performance testing?Why do performance testing?
• May not detect some functional defects that onlyMay not detect some functional defects that only
appear under load.appear under load.
• Does the application respond quickly enough forDoes the application respond quickly enough for
the intended users?the intended users?
• Will the application handle the expected user• Will the application handle the expected user
load and beyond?load and beyond?
• Will the application handle the number of• Will the application handle the number of
transactions required by the business?transactions required by the business?
• Is the application stable under expected and• Is the application stable under expected and
unexpected user loads?unexpected user loads?

Performance Tester Must KnowPerformance Tester Must Know
• What is the type of communication between the client and server?What is the type of communication between the client and server?
• Requirements to monitor & Transaction Response Time or serverRequirements to monitor & Transaction Response Time or server
utilization (SLA).utilization (SLA).
• Is the application client-side?, And the AUT architecture.Is the application client-side?, And the AUT architecture.
• Is there client-side activities involved in the entire application?Is there client-side activities involved in the entire application?
• What is the authentication mechanism?What is the authentication mechanism?
• Does the application allow multiple login of a single user?Does the application allow multiple login of a single user?
• How does the application maintain a session?.How does the application maintain a session?.
• Environment specifications including Software and hardware.Environment specifications including Software and hardware.
• How many users or Volumetric client expecting?How many users or Volumetric client expecting?
• How does the load balancer distribution the load?How does the load balancer distribution the load?
• What are the parameters been sent to the Server?What are the parameters been sent to the Server?

Loadrunner BenifitsLoadrunner Benifits
• HP LoadRunner supports performance testing for a wide range ofHP LoadRunner supports performance testing for a wide range of
application environmentsapplication environments and protocolsand protocols
• Decreases the risk of deploying systems that do not meet performanceDecreases the risk of deploying systems that do not meet performance
requirementsrequirements
• Reduces hardware and software costs by accurately predicting systemReduces hardware and software costs by accurately predicting system
capacity.capacity.
• Load Runner reduces the personnel requirementsLoad Runner reduces the personnel requirements
b) Load Runner reduces the hardware requirementsb) Load Runner reduces the hardware requirements
c) The controlling will be accomplished effectively by the controllerc) The controlling will be accomplished effectively by the controller
d) Because Load Runner tests are fully automated, we can easily repeatd) Because Load Runner tests are fully automated, we can easily repeat
them as many times we needthem as many times we need
• Lets you begin intelligent service-level management before services go liveLets you begin intelligent service-level management before services go live
• Shortens test cycles to accelerate delivery of high quality applicationsShortens test cycles to accelerate delivery of high quality applications
• Pinpoints end-user, system-level and code-level bottlenecks rapidly andPinpoints end-user, system-level and code-level bottlenecks rapidly and
with ease.with ease.
• Reduces the cost of defects by early testing in the development cycleReduces the cost of defects by early testing in the development cycle

Loadrunner VersionsLoadrunner Versions
• LR Version 6.0--- Used a separate thread per user, which required almost 10 timesLR Version 6.0--- Used a separate thread per user, which required almost 10 times
more i/o and CPU cycles than 6.5.more i/o and CPU cycles than 6.5.
• Version 6.5 available June 2000 –Turbo Load technology-thousands of vusers using aVersion 6.5 available June 2000 –Turbo Load technology-thousands of vusers using a
single operating system thread.single operating system thread.
• Version 7.8 became available September 2003.Version 7.8 became available September 2003.
• Version 7.8 Feature Pack 1 added support for Windows XP.Version 7.8 Feature Pack 1 added support for Windows XP.
• Version 8.0 became available August 2004 –Additional Runtime Settings & DiagnosticsVersion 8.0 became available August 2004 –Additional Runtime Settings & Diagnostics
and tuning Capabilities.and tuning Capabilities.
• Version 8.1 became available October 2005– Workflow view and workflow wizardVersion 8.1 became available October 2005– Workflow view and workflow wizard
renamed as Execution log and replay log.renamed as Execution log and replay log.
• Version 8.1 Feature Pack 3 –Web services Enhancements.Version 8.1 Feature Pack 3 –Web services Enhancements.
• Version 8.1 Feature Pack 4—Upgraded recorder version.Version 8.1 Feature Pack 4—Upgraded recorder version.
• Version 8.1 Feature Pack 4 --January 2, 2007.Version 8.1 Feature Pack 4 --January 2, 2007.
• Version 9.10 installer, available Feb. 2008.Version 9.10 installer, available Feb. 2008.
• Version 9.50 available June 2009.Version 9.50 available June 2009.
• Version 9.51 patch to 9.50 --July 6, 2009, enables better AJAX recognition andVersion 9.51 patch to 9.50 --July 6, 2009, enables better AJAX recognition and
specification of DOM element properties. which require Vista users to allow.specification of DOM element properties. which require Vista users to allow.

Features of Load runner 9.5Features of Load runner 9.5
• Protocol Advisor :Protocol Advisor : Protocol Advisor helps you to determine an appropriateProtocol Advisor helps you to determine an appropriate
protocol for recording a Vuser script. The Protocol Advisor scans yourprotocol for recording a Vuser script. The Protocol Advisor scans your
application for elements of different protocols and displays a list of theapplication for elements of different protocols and displays a list of the
detected protocols.detected protocols.
• We can export the Test Result from VUGen scripts to HTML and openWe can export the Test Result from VUGen scripts to HTML and open
Quality Center defects directly from the report.Quality Center defects directly from the report.
• Load Runner 9.5 supports Windows Vista SP1.Load Runner 9.5 supports Windows Vista SP1.
• Provides support uptoProvides support upto .NET Framework 3.5.NET Framework 3.5 versionversion and supports .NETand supports .NET
and Web Services protocols.and Web Services protocols.
• LoadRunner is now integrated with a 3rd party softwareLoadRunner is now integrated with a 3rd party software Shunra,Shunra, to provideto provide
you access to WAN emulation.you access to WAN emulation.
• LR controller has a new utility calledLR controller has a new utility called Host Security Manager utilityHost Security Manager utility thatthat
will allow you to create secure communication channels between thewill allow you to create secure communication channels between the
LoadRunner Controller and load generators.LoadRunner Controller and load generators.
• There is a new Analysis API that will allow you to create an analysis sessionThere is a new Analysis API that will allow you to create an analysis session
from test resultsfrom test results
• LR analysis now supportsLR analysis now supports SQL Server 2005SQL Server 2005 ..
• LoadRunner 9.5 no longer supports Windows 2000.LoadRunner 9.5 no longer supports Windows 2000.
• LR 9.5 is compatible with Quality Center QC) 9.2, 10.0 and QTP 9.5 andLR 9.5 is compatible with Quality Center QC) 9.2, 10.0 and QTP 9.5 and
10.0 versions.10.0 versions.

Key Types of Performance TestingKey Types of Performance Testing
• Performance test :Performance test :
A performance test is a technical investigation done toA performance test is a technical investigation done to
determine or validate the response, speed, scalability,determine or validate the response, speed, scalability,
stability characteristics of the product under test.stability characteristics of the product under test.
• Focuses on determining if the user of the system will beFocuses on determining if the user of the system will be
satisfied with the performance characteristics of thesatisfied with the performance characteristics of the
application.application.
• Identifies mismatches between performance-relatedIdentifies mismatches between performance-related
expectations and reality.expectations and reality.
• Supports tuning, capacity planning, and optimizationSupports tuning, capacity planning, and optimization
efforts.efforts.
PurposePurpose :: To determine or validate speed, scalability, and/or stability.To determine or validate speed, scalability, and/or stability.

• Load test:Load test:
A load test enables you to measure response times, throughputA load test enables you to measure response times, throughput
rates, and resource-utilization levels, and to identify yourrates, and resource-utilization levels, and to identify your
application’s breaking point, assuming that the breaking pointapplication’s breaking point, assuming that the breaking point
occurs below the peak load condition.occurs below the peak load condition.
 Performance objectives are often specified in a service levelPerformance objectives are often specified in a service level
agreement (SLA).agreement (SLA).
PurposePurpose::
To verify application behavior under normal and peak loadTo verify application behavior under normal and peak load
conditions.conditions.
• Determines the sufficiency of a hardware environment.Determines the sufficiency of a hardware environment.
• Evaluates the adequacy of a load balancer.Evaluates the adequacy of a load balancer.
• Detects concurrency issues.Detects concurrency issues.
• Determines how many users can application handle.Determines how many users can application handle.
• Detects functionality errors under load.Detects functionality errors under load.

• Stress testStress test ::
SStress testing enables you to identify your application’s weak points,tress testing enables you to identify your application’s weak points,
and shows how the application behaves under extreme load conditions.and shows how the application behaves under extreme load conditions.
Purpose:Purpose:
 To determine or validate an application’s behavior when it is pushed beyond normalTo determine or validate an application’s behavior when it is pushed beyond normal
or peak load conditions.or peak load conditions.
 Determines the side effects of common hardware or supporting application failures.Determines the side effects of common hardware or supporting application failures.
 Determines if data can be corrupted by overstressing the system.Determines if data can be corrupted by overstressing the system.
 Helps to determine what kinds of failures are most valuable to plan for.Helps to determine what kinds of failures are most valuable to plan for.
Challenges :Challenges :
 Because stress tests are unrealistic by design, some stakeholders may dismiss testBecause stress tests are unrealistic by design, some stakeholders may dismiss test
results.results.
 It is often difficult to know how much stress is worth applying.It is often difficult to know how much stress is worth applying.
• Capacity test:Capacity test:
 Capacity testing helps you to identify a scaling strategy in order toCapacity testing helps you to identify a scaling strategy in order to
determine whether you should scale up or scale Down.determine whether you should scale up or scale Down.
 Capacity testing is conducted in conjunction with capacity planning, which you use toCapacity testing is conducted in conjunction with capacity planning, which you use to
plan for future growth, such as an increased user base or increased volume of data.plan for future growth, such as an increased user base or increased volume of data.
For example, to accommodate future loads, you need to know how many additionalFor example, to accommodate future loads, you need to know how many additional
resources (such as processor capacity, memory usage, disk capacity, or networkresources (such as processor capacity, memory usage, disk capacity, or network
bandwidth) are necessary to support future usage levels.bandwidth) are necessary to support future usage levels.

SOAK TEST/EnduranceSOAK TEST/Endurance
TestTest
• Soak testing involves testing a system with aSoak testing involves testing a system with a
significant load extended over a significantsignificant load extended over a significant
period of time, to discover how the systemperiod of time, to discover how the system
behaves under sustained use.behaves under sustained use.
• For Example: a system may behave exactly asFor Example: a system may behave exactly as
expected when tested for 1 hour. However,expected when tested for 1 hour. However,
when it is tested for 3 hours, problems such aswhen it is tested for 3 hours, problems such as
memory leaks cause the system to fail ormemory leaks cause the system to fail or
behave randomly.behave randomly.

Performance Testing ApproachPerformance Testing Approach

Performance Test ApproachPerformance Test Approach

Performance Testing ToolsPerformance Testing Tools
• Load RunnerLoad Runner
• Rational RobotRational Robot
• Silk PerformerSilk Performer
• Web LoadWeb Load
• OpenSTAOpenSTA
• Visual Studio Team SystemVisual Studio Team System
• JMeterJMeter

• Comparision of features of Loadrunner, Silk Performer and RPTComparision of features of Loadrunner, Silk Performer and RPT
• FeaturesFeatures
Sl no 1: Protocol SupportSl no 1: Protocol Support
Load Runner :Load Runner : Supports over 51 protocolsSupports over 51 protocols
Silk Perfomer:Silk Perfomer: Supports around 40 protocolsSupports around 40 protocols
RPT :RPT :Very few - HTTP, SAP, Sockek API, CitrixVery few - HTTP, SAP, Sockek API, Citrix
Sl no 2 : Selection of multiprotocol for scriptingSl no 2 : Selection of multiprotocol for scripting
Load Runner :Load Runner : SupportedSupported
Silk Perfomer:Silk Perfomer: Not supportedNot supported
RPT :RPT : Not supportedNot supported
Sl no 3 : Data corelationSl no 3 : Data corelation
Load Runner :Load Runner : Much simpler and easierMuch simpler and easier
Silk Perfomer :Silk Perfomer : ClumsyClumsy
RPT :RPT : SimplerSimpler
Sl no 4 : Fune tuningSl no 4 : Fune tuning
Load Runner :Load Runner :HP Sitescope and Diagnostics available for troubleshooting the identified bottlenecksHP Sitescope and Diagnostics available for troubleshooting the identified bottlenecks
Silk Perfomer:Silk Perfomer: Not possible, rely on 3rd party toolNot possible, rely on 3rd party tool
RPT :RPT :To use IBM's TivoliTo use IBM's Tivoli
Sl no 5 : Identifiation of bottlenecksSl no 5 : Identifiation of bottlenecks
Load Runner :Load Runner :Easier and fasterEasier and faster
Silk Perfomer:Silk Perfomer: Not easyNot easy
RPT :RPT :Not easyNot easy
Sl no 6 : AnalysisSl no 6 : Analysis
Load Runner :Load Runner :Around 100 types garphs present. Cross garph result.Help in finding the bottle neck.Around 100 types garphs present. Cross garph result.Help in finding the bottle neck.
Silk Perfomer:Silk Perfomer: Supports cross graphsSupports cross graphs
RPT :RPT :Has limitationHas limitation
Sl no 7 : Analysis reportSl no 7 : Analysis report
Load Runner :Load Runner :Repors - html,excel and wordRepors - html,excel and word
Silk Perfomer:Silk Perfomer: Only htmlOnly html
RPT :RPT :Only htmlOnly html
Sl no 8 : Industry-standard scripting languageSl no 8 : Industry-standard scripting language
Load Runner :Load Runner : Load Runner:C,Java,VBLoad Runner:C,Java,VB
Silk Perfomer:Silk Perfomer: bdlbdl
RPT :RPT : JavaJava
Sl no 9 : Support levelSl no 9 : Support level
Load Runner :Load Runner : Three levels of support available. Good support providedThree levels of support available. Good support provided
Silk Perfomer:Silk Perfomer: Not too good in providing supportNot too good in providing support
RPT :RPT : Not too good in providing supportNot too good in providing support
Sl no 10 : Latency Time analysisSl no 10 : Latency Time analysis
Load Runner :Load Runner : End to end response time as well as response time at each level supported.End to end response time as well as response time at each level supported.
Silk Perfomer:Silk Perfomer: Displays response time at end to end as well as in the method levelDisplays response time at end to end as well as in the method level
RPT :RPT : Displays response time end to endDisplays response time end to end
Sl no 11 : Distributed Load GenerationSl no 11 : Distributed Load Generation
Load Runner :Load Runner : Can control multiple load generators and collect results. Can control load generators located at remote networks (through a firewall) if requiredCan control multiple load generators and collect results. Can control load generators located at remote networks (through a firewall) if required
Silk Perfomer:Silk Perfomer: Single MMC can control upto 256 agents (load injectors) provided sufficient physical memory is available on the MMC.Single MMC can control upto 256 agents (load injectors) provided sufficient physical memory is available on the MMC.
RPT :RPT : Not SupportedNot Supported

Understanding LoadRunner LicensingUnderstanding LoadRunner Licensing
Load Runner licenses fall into one of the following categories:Load Runner licenses fall into one of the following categories:
• permanent license-Annual maintenance fee requiredpermanent license-Annual maintenance fee required
• Term license -either 1 month or 3 monthsTerm license -either 1 month or 3 months
• Virtual User Days –ex:1000 vusers for one dayVirtual User Days –ex:1000 vusers for one day
• Software as a Service :- Hp will provide the controller and chargeSoftware as a Service :- Hp will provide the controller and charge
When you purchase a Load Runner license, you will need to purchase a license for aWhen you purchase a Load Runner license, you will need to purchase a license for a LoadLoad
Runner ControllerRunner Controller and for a certain number ofand for a certain number of Virtual UsersVirtual Users of a particular type.of a particular type.

Performance Testing ProcessPerformance Testing Process
The following steps are involved in load-testing:The following steps are involved in load-testing:
• Step 1 - Identify performance acceptanceStep 1 - Identify performance acceptance
criteriacriteria
• Step 2 - Identify key scenariosStep 2 - Identify key scenarios
• Step 3 - Create a workload modelStep 3 - Create a workload model
• Step 4 - Identify the target load levelsStep 4 - Identify the target load levels
• Step 5 - Identify metricsStep 5 - Identify metrics
• Step 6 - Design specific testsStep 6 - Design specific tests
• Step 7 - Run testsStep 7 - Run tests
• Step 8 - Analyze the resultsStep 8 - Analyze the results

Loadrunner Testing ProcessLoadrunner Testing Process

Performance and TechnicalPerformance and Technical
Testing ApproachTesting Approach

Load testing typically consists of five phases: planning,Load testing typically consists of five phases: planning,
script creation, scenario definition, scenario execution,script creation, scenario definition, scenario execution,
and results analysis.and results analysis.
Define your performance testing requirements, forDefine your performance testing requirements, for
example, number of concurrent users, typical businessexample, number of concurrent users, typical business
processes and required response times.processes and required response times.
Create Vuser Scripts:Create Vuser Scripts: Capture the end-user activities intoCapture the end-user activities into
automated scripts.automated scripts.
Define a Scenario:Define a Scenario: Use the LoadRunner Controller to setUse the LoadRunner Controller to set
up the load test environment.up the load test environment.
Run a Scenario:Run a Scenario: Drive, manage, and monitor the load testDrive, manage, and monitor the load test
from the LoadRunner Controller.from the LoadRunner Controller.
Analyze the Results:Analyze the Results: Use LoadRunner Analysis to createUse LoadRunner Analysis to create
graphs and reports, and evaluate the performance.graphs and reports, and evaluate the performance.
Loadrunner Testing ProcessLoadrunner Testing Process

LoadRunner Internal Architecture

Keylabs Training.comKeylabs Training.com
Key for IT SolutionsKey for IT Solutions
• QA TrainingQA Training

Performance testing material

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Performance testing material (20)

Recently uploaded (20)

Performance testing material