Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testing projects
2 likes693 views
Liang Gao, Huawei, Trevor Cooper, Intel NFV environments are highly flexible and this introduces unique challenges for testing performance of NFVI and Network Services. This presentation introduces OPNFV performance test projects and explains their role as part of the testing ecosystem. Examples from three performance testing categories will be demonstrated showing test results and their interpretation. Test cases discussed will include data-path performance, live migration performance and storage performance.
More Related Content
What's hot (20)
Viewers also liked (20)
Similar to Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testing projects (20)
More from OPNFV (14)
Recently uploaded (20)
Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testing projects
- 2. Evaluate NFVI Performance by Leveraging OPNFV Testing Projects Gao Liang, Huawei Trevor Cooper, Intel Corp.
- 3. Agenda 1. Challenges 2. Rules & Choices 3. Examples & Demo 4. Future Directions
- 4. NFV Performance Testing Challenges • Complex SUT configurations are required for optimal performance • Numerous choices are required for test tools as well as test environments • Virtual applications and network NFVI resources may interact • Multi-vendor solutions add to testing complexity • Deployment/orchestration automation impacts performance • Testing methods must be integrated with CI and DevOps Traditional Performance Testing Methods are Useful but Not Sufficient
- 5. “Rules” for Efficient NFV Performance Testing 1. Know yourself … developer, solution architects, user perspectives differ 2. Know your use-case … service and network SLAs define workloads 3. Know your words … metric definitions and test methods impact results 4. Know your friends … OPNFV and other communities can help you
- 7. OPNFV Performance Testing Menu 1. System Under Test (SUT) 2. Traffic Generator 3. Test Automation Framework NFVI + VIM + MANO • SampleVNF (vACL, vFW, vCGNAT, …) • SFC, … Commercial hardware • Ixia • Spirent • Xena NFVI + VIM + MANO • Dovetail • NFVbench NFVI + VIM • Compute • Storage • Network • vloop-vnf (dpdk-testpmd, Linux bridge, L2fwd module) • Spirent stress-VM • Virtual Traffic classifier Commercial virtual • Ixia • Spirent NFVI + VIM • Yardstick • Storperf • Bottleneck • Qtip NFVI • Virtual switching (OVS, OVS-dpdk, VPP) • HW offload (TSO, crypto, SmartNIC) • Physical / virtual interfaces • NIC (10GE, 40GE, …) • Vhost-user • Pass-through, SR-IOV Open Source software • Pktgen • Moongen • TREX • PROX NFVI • VSPERF • Storperf Drive standardization Promote a defined platform and reuse Establish best practice
- 8. Integrate Deploy Test Deploy Test Master (N) Release Validation NFVI components & subsystems Stable (N-1) Stress Testing Project Master (N) Merge & Verify OPNFV Dev CI/CD/CT Project Stable (N-1) Daily Test Cases Test Scripts Configurations Performance Ranges Pre-production Evaluation Operator Proprietary Tests Test Data Test Artifacts Scenarios OPNFV Production CI/CD/CT Compliance Testing Test Dashboards Reference Performance Ranges Reference Configurations Euphrates Performance Testing NFVI + VIM Dashboards User NFVI Testing Commercial VNF Testing OperatortestingVendortesting Developertesting Release testing
- 9. VSPERF DUT is an important part of the E2E Data Path • Virtual Switching technology and offloads • Physical and virtual ports • Virtualized Workload 9 VSPERF Test Automation • Source/build SW components • Set up vSwitch • Set up workload • Set up traffic generator • Execute test cases • Collect test results • Log and store data • Generate test statistics & result dashboards / reports Network SLA Capacity / BW Loss Delay Delay variation Network Performance Metrics & Statistics
- 10. VSPERF Dataplane Performance Testing Switching Operations • Speed • Accuracy • Reliability • Scalability NFVI Forwarding Capability • Frame loss • Maximum forwarding rate • Burst behavior • Packet Delay Distribution • Scalability (flows, active ports, etc.)
- 11. Yardstick Overview •Methodology for verifying infrastructure from the perspective of a VNF aligned with ETSI TST 001. Test VM Test VM … Define Identify Select RUN •Where do we fit E2E NFV testing NFVI Testing VNF Testing Pre- Deployme nt Testing VNF infra Testing Yardstick Collect
- 12. Yardstick Test case Overview Storage Storage IOPS thrp, latency disk size, block size and disk utilization Network * Network Latency between VM * Network throughput and packet loss * Packet delay variation * Network throughput and packet loss * Network latency in dpdk * Network latency between nodes * Cache hit/miss ratio and usage, nw thrp and latency * Network capacity and scale * IP datagram error rate, ICMP message error rate, TCP segment error rate and UDP datagram error rate Compute Cache utilization memory read latency memory rw bandwidth Processing speed,CPU Load Meminfo, CPUinfo IPv6 IPV6 network latency KVM4NFV KVM Latency measurements HA Openstack Service HA Neutron Server Keystone Glance Api Cinder Api Swift Proxy Network HA CPU Overload HA Disk I/O Block HA Load Balance Service HA Openstack Virtual IP HA Parser translate from yang to tosca Bottlenecks Posca stress traffic (throughput) Posca stress ping (life cycle) Vsperf rfc2544_back2back_dummy rfc2544_continuous_dummy rfc2544_throughput_dummy Storperf Storage performance via Storperf Feature Test Case NFVI extended CPU Pinning VM support NUMA Live Migration Cold Migration Huge pages VLAN/VxLAN Affinity and Anti-affinity SR-IOV DPDK NFV extended NFVI Generic Integrating test caseNetwork Service Benchmark
- 13. Yardstick Architecture Overview Extensible •Restful API & CLI support ，can be easily integrated •Plug-in support , can easily integrate other tools •Python 3 support Customizable •Core as a Orchestrator •Scenarios as a library (Test operation) •Contexts as a library (Test Pre-condition) •Jinja2 support (Test case) Ease to Use •Docker image •Offline support •UI &DB and Report
- 14. • Using Yardstick test cases to evaluate NFVI performance • Installation • Web UI • SUT management • Test case management • Project->Task->report • Live Migration Test case Run • Upload your Yardstick test case to evaluate NFVI performance • Upload test case • Run storperf test case via Yardstick. Yardstick Demo
- 15. Future • Advanced Stress Testing • Simulator capabilities • Post processing/analytics/staging • Data + modeling • Performance Testing as a Service • Micro Service Approach • Run any test case from any projects you want with Unified User Interface • Common Traffic Generator, VNF on boarding, System level configuration. • All projects with Common DB , Test API, Dashboard