Automated Testing of NASA Software
Download as pptx, pdf4 likes2,635 views
This document summarizes an approach to automated testing of large, multi-language software systems using cloud computing. It discusses applying a lightweight, model-based test generation and execution approach to several NASA projects, including GMSEC, Core Flight Software, Space Network, and Mars Science Laboratory. Models of system APIs and interfaces are developed and used to automatically generate test cases, finding bugs. The approach has been successfully transferred to other teams and is being applied to additional complex NASA systems.
More Related Content
What's hot (20)
Similar to Automated Testing of NASA Software (20)
More from Dharmalingam Ganesan (20)
Recently uploaded (20)
Automated Testing of NASA Software
- 1. CESE Automated Testing of Large Multi- language SW Systems using Cloud Computing Executive Status Report Principal Investigator (PI): Dr. Mikael Lindvall, CESE NASA POC: Markland Benson, White Sands Team members: Dharma Ganesan, Dr. Chris Ackermann(CESE) GMSEC, CFS, Space Network, MSL © 2011 Fraunhofer USA, Inc. 1 Center for Experimental Software Engineering
- 2. CESE Motivation • Bugs can lead to deaths, injuries, or financial loss • Software testing is necessary to find the presence of bugs • Software testing consumes 50% - 75% of the development effort • Many NASA projects could benefit from automated test generation • In fact, several skeptical engineers noted that the proposed work is illuminating! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
- 3. CESE Technical Context • The system(s) under test (SUT) are complex • Multiple programming languages – Ada, C, C++, Fortran, Java • Distributed legacy systems – More than 20 yrs old – Runs on multiple machines and involves various OSes,e.g. OpenVMS • GUI intensive – Web, Java Swing, AWT, X Windows, Motif © 2011 Fraunhofer USA, Inc. 3 Center for Experimental Software Engineering
- 4. CESE Problems • Test cases are developed manually • Some test execution is automated (e.g., JUnit) • Test cases miss valid “corner” cases • Test cases are also programs: not easy for non- technical stakeholders to understand • Difficult to summarize what was tested Approach: Lightweight Model-based Test Generation and Execution © 2011 Fraunhofer USA, Inc. 4 Center for Experimental Software Engineering
- 5. CESE Approach • Develop a model of the system under test • The model contains actions and expected results • Automatically derive test cases from the model • Execute the test cases • Decide when to stop testing based on model coverage! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
- 6. CESE An Example Model APIs of the module under test public interface IConnection { public Status Connect(); public Status Disconnect(); © 2011 Fraunhofer USA, Inc. } Center for Experimental Software Engineering
- 7. CESE Currently Targeted Projects • GMSEC – Reusable framework for ground systems – Modeled the Core API and Applications – Generated executable test cases from the model – Confirmed defects/violations reported and fixed • Core Flight Software – Reusable framework for flight systems – Modeled the OS abstraction layer (OSAL) – Generated executable test cases from the model – Confirmed defects/violations reported and fixed – Test cases delivered to the CFS team © 2011 Fraunhofer USA, Inc. 7 Center for Experimental Software Engineering
- 8. CESE Currently Targeted Projects • Space Network – White Sands – Evaluating different test execution tools suitable for Open VMS environment – E.g. Image-based test execution tools: EggPlant and Sikuli – Challenges in this project: • SUT not possible to run at Fraunhofer • Must visit Goddard with an escort (secured building) • Not possible to connect to Internet © 2011 Fraunhofer USA, Inc. 8 Center for Experimental Software Engineering
- 9. CESE Currently Targeted Projects • Mars Science Laboratory (MSL) – JPL – Can current test cases be modeled and used to generate new test cases? • New set cover scenarios not previously covered? • New approach more cost effective? – Challenges in this project: • SUT not possible to run at Fraunhofer • Software is sensitive (ITAR) © 2011 Fraunhofer USA, Inc. 9 Center for Experimental Software Engineering
- 10. CESE Current Results • An end-to-end approach for test generation – Successfully used on the GMSEC and the CFS and detected bugs • Approach works well on different levels: – API (Module interface) level testing – GUI testing • Approach transferred to an intern at GSFC – The intern found new bugs using the approach – Demonstrates the approach is lightweight © 2011 Fraunhofer USA, Inc. 10 Center for Experimental Software Engineering
- 11. CESE Summary and Next Steps • We’re building a new approach that – Helps in test automation using lightweight models • Prototype approach successfully applied to GMSEC and CFS/cFE • Next steps are to apply to the Space Network and the MSL projects • Examples and technical details in technical presentation! © 2011 Fraunhofer USA, Inc. 11 Center for Experimental Software Engineering