Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors
2 likes1,908 views
VNF sizing is one of the big issues for users, integrators and also venders/providers of VNF. In legacy system, network functions are integrated in the systems with carefully sizing by estimated performance metrics, but in NFV, performance and metrics of VNF are depends on flavors. To decide which flavor is suitable for requirements, we have to know the basic data of each metrics with variable flavors. But there are many parameters, so automatic testing is necessary. We had developed the test automation system of VNF that can automatically conducting the sequence of tests with variable parameters such as number of core, memory size, interface speed, etc. We already created the test cases that evaluate BGP interoperability of virtual router and performance test of Cisco, Juniper, Brocade, and so on. We think the activity is similar to Yardstick, so we would like to collaborate with them. In this talk, we summarize our activities and development result, and introducing our latest plan to collaborate with OPNFV.
More Related Content
What's hot (20)
Viewers also liked (20)
Similar to Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors (20)
More from OPNFV (20)
Recently uploaded (20)
Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors
- 1. June 20–23, 2016 | Berlin, Germany
- 2. Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors Takashi Torii NEC Corporation Tatsuya Tohyama, Okinawa Open Laboratory/OKIT
- 3. Introduction of Okinawa Open Laboratory(OOL)
- 4. Vision: Okinawa Smart Hub • Okinawa Prefectural Government vision and policy for industrial development and promotion • Attract companies, talents and knowledge from other prefectures in Japan and overseas, especially Asian countries, and promote new value creation in Okinawa • Form major international ICT hub in Asia
- 5. Okinawa Smart Hub and OOL • Contribute to realizing Okinawa Smart Hub concept • Based in Okinawa with support from Okinawa Prefectural Government, attracting leading companies, organizations, educational institutions, and research bodies from other prefectures in Japan and overseas, especially Asia, and promoting international R&D activities • Contributes to promoting concentration of ICT industries and advancement of regional companies SDN technology research and standardization base OSS cloud development community Leading Linux and OSS organization OSS SDN development project Okinawa Open Laboratory Practical-use research base that integrates cloud technology and SDN technology International R&D organization Leading companies and organizations from other prefectures in Japan and overseas, especially Asia Technology cooperation Cooperation, attraction Activity support ・Support for R&D activities ・Promotion of cloud DC facility use
- 6. Mission • Research and development activities for practical application and dissemination of advanced Information Communications Technologies (ICT), especially next-generation ICT platform technologies, such as • Software-Defined Networking (SDN) • Cloud Computing using Open Source Software (OSS, especially Open Stack) • Open Innovations • Collaboration, Joint Work • Bridging the gaps between users and providers
- 7. Activity Model • OOL connects OSS development communities and market via use cases and reference models to expand user base of next-generation ICT platform technologies in cooperation with MEMBER and partners • OOL Value Chain: Use Case à Verification à Ref. Model à Promotion • Feedback Loops: (1) OOL ó Market, (2) OOL ó OSS Dev. Communities OSS Bodies (Community/Consortium) Develop Use Case Verify Promote Issues /Needs Apply /Use OOL and its Members Market (Providers and End Users) Expand User Base Reference Model Testbed (OpenStack with SDN)
- 9. Project Formations Full Time Engineers Organizer Companies Research Institute Academia • Planning projects and goals • Communicate with companies • Communicate with Boards, Staffs and Stakeholders (Government etc) • Manage the activities • Promote and collaborate Staffs Board
- 10. Dev Project Example: OF-Patch • Patch panel function implemented using OpenFlow technology OFPM User Change network topology by remote operation OF-Patch GUI OFC OFS Patch by flow Host1 Host2 Host3
- 11. OPNFV Test Lab • VNF Test Automation • Service Function Chaining • Joint POC with oversea members Internet Jump Server POD#1 (Fuel) POD#2 (RDO)
- 12. VNF Test Automation Project
- 13. Background • Market of VNF is now emerging • ETSI proposed NFV architecture • Open Source VNF • VyOS (Vyatta） • Open vSwitch …etc • Vender VNF • Cisco CSR1000V • Juniper FireFly, vSRX • Brocade Vyatta Core …etc • But we don‘t have test automation platform of VNF
- 14. Background • Problem of VNF testing • There is no standard • Performance and stability is depend on flavors • Many test patterns are required than H/W appliance • Impossible to cover all test patterns by manual • Risk of human error or miss configuration • Configurations are depend on vendors • Need support from venders or skilled engineers in all phase • To solve these problems, we have developed VNF automation testing system
- 15. How to test VNF? • Workflow • Deploy • Deploy VNF on VIM with flavor • Initialize VNF • Configuration • Configure and confirm • Check the network connectivity with other devices • Run • Subscribe test patterns and run on traffic generator. • Check VNF status and behavior • Reporting • Writing report based on result
- 17. Test workflow • Benefit of automation • Engineer can focus on troubleshooting • Engineers who don’t have much skills can operate the tests • High repeatability Repeat Test Deploy VNF Initialize Initialize VNF Config Delete VNF Log Archive Analysis ReportCheck
- 18. Architecture PoC User Orchestrator Web-Portal Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA)
- 19. Architecture PoC User Orchestrator Web-Portal Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA)
- 20. Web Portal • Set the test parameters • Select the test scenario from drop down lists Orchestrator Web-Portal REST API PoC User Test Parameter
- 21. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
- 22. Orchestrator • Controller management • Controls each components based on the configuration of test scenarios. • Creating test scenario • Create test scenarios by GUI • Reporting • Output PDF format Scenario generate tool
- 23. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
- 24. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
- 25. Virtual Appliance Controller • Multi virtualized platform control • Deploy virtual appliance on selected virtualized platform. Orchestrator REST-API REST-Message OpenStack Link Virtual Appliance Controller REST-API REST-API
- 26. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
- 27. NW Device Controller • Multi vendor control • Provides the ability to control multiple vendor VNF Orchestrator NW Device Controller REST-API Telnet/SSH/Netconf REST-Message Parameter Ex ・Vender Name ・Vender Version ・Test scenario other…
- 28. Architecture PoC User Orchestrator Web-Portal Report generationManagement of Test REST API Interoperability Traffic Test REST API Virtual Appliance Ctr (Deploy of VNF appliance) NW Device Ctr (Insert/delete of configure) Tester Ctr (Control of IXIA) Web-Portal
- 29. Tester Controller • Generating Tcl script • Based on the parameters set by REST API from Orchestrator • Ixia Driver • Load Tcl Script by IXIAAPI. Orchestrator Tester Controller REST-Message Parameter Ex ・Interface Information ・Packet Size(Short,Long,Imix) ・Frame rate other... Test Traffic IXIA API REST API Tcl Script
- 31. Our environment VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
- 32. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
- 33. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance test environment VNF appliance test environment
- 34. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance for interoperability testing VNF appliance for interoperability testing
- 35. Our environment for interoperability VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
- 36. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
- 37. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 VNF appliance test environment VNF appliance test environment
- 38. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600 Traffic Generator
- 39. Our environment for Traffic Test VyOS vSwitch vnic0 vSwitch InteroperabilityL2SW TrafficTestL2SW vnic1 vnic1 OpenStack (node2) VMware Esxi (node1) VMware Esxi (node3) OpenStack (node4) vnic2 vnic3 vnic1 vnic2 vnic3 MGMT-Seg vnic0 vnic0 vSwitch Juniper Firefly Cisco CSR1000v Vyatta 5600 vnic0 vSwitch vnic1 VyOS Juniper Firefly Cisco CSR1000v Vyatta 5600
- 41. Result of test automation • Enables multiple patterns of test runs which are difficult by manual Automate the combination test of 2736 patterns by 40 hours 【CSR1000v,Firefly,Vyatta,VyOS】VNF appliance 【VMware ESXi, OpenStack】VIM 【CPU,Mem,HDD,VNIC】 【CSR1000v,Firefly,Vyatta,VyOS】Interoperability with reference router 【VMware ESXi, OpenStack】VIM of reference router 【BGP:5 Pattern,OSPF:7 Pattern】 【Test Pattern:3 Pattern ,FrameRate:6 Pattern】 Flavor of VNF Appliance Routing protocol Traffic test pattern
- 42. Current Activities • We have just joined OPNFV, and deploy Community Lab. • Refactoring to align NFV Architecture of ETSI • Using TOSCA based orchestrator (we are considering Tacker) • Collaborate with Functest
- 44. Any Questions?