Software Architecture Evaluation: A Systematic Mapping Study
1 like353 views
Sofia Ouhbi presented a systematic mapping study on software architecture evaluation approaches at the 13th International Conference on Evaluation of Novel Approaches to Software Engineering in Funchal, Madeira, Portugal. The study analyzed 60 papers on software architecture evaluation to identify key publication sources, trends over time, research types, validation methods, evaluation approaches, quality models used, and description models. The results showed that journals and conferences are the main publication sources, interest peaked in the late 2000s and has declined recently, case studies are common validation methods, and quality attributes like performance and maintainability are frequently evaluated using models like the ISO 9126.
More Related Content
What's hot (20)
Similar to Software Architecture Evaluation: A Systematic Mapping Study (20)
More from Sofia Ouhbi (8)
Recently uploaded (20)
Software Architecture Evaluation: A Systematic Mapping Study
- 1. + 13th International Conference on Evaluation of Novel Approaches to Software Engineering. Funchal, Madeira, Portugal 23 March, 2018 Presented by: Sofia Ouhbi Software Architecture Evaluation A systematic mapping study Sofia Ouhbi ENASE'18. Madeira, Portugal.
- 3. + Introduction Software Architecture ENASE'18. Madeira, Portugal. 3
- 4. + A software architecture is: “the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both” “All architecture is design but not all design is architecture.” Grady Booch (2006) ENASE'18. Madeira, Portugal. P.Clements et al.,Documenting Software Architectures:Views and Beyond, 2nd ed.,Pearson Education,2010. 4 • Architecture:What and where? • Design: How?
- 5. + Software Architecture ENASE'18. Madeira, Portugal. The complexity of man-made systems has grown to an unprecedented level.This has led to new opportunities, but also to increased challenges for the organizations that create and utilize systems. Concepts, principles and procedures of architecting are increasingly applied to help manage the complexity faced by stakeholders of systems. ISO/IEC/IEEE 42010:2011 Systems and software engineering -- Architecture description 5
- 6. + Software Architecture ENASE'18. Madeira, Portugal. During the mid-1990s, software architecture started to emerge as a broader discipline that involved the study of software structures and architectures in a more generic way. 6
- 7. + Software Architecture ENASE'18. Madeira, Portugal. The architecture is not the operational software. Rather, it is a representation that enables you to: (1) analyze the effectiveness of the design in meeting its stated requirements, (2) consider architectural alternatives at a stage when making design changes is still relatively easy, (3) reduce the risks associated with the construction of the software. Roger S.Pressman 7
- 8. + Software Architecture ENASE'18. Madeira, Portugal. n Software architecture provides a representation that facilitates communication among all stakeholders. n The architecture highlights early design decisions that will have a profound impact on all software engineering work that follows. n Architecture “constitutes a relatively small, intellectually graspable model of how the system is structured and how its components work together” Bass,L.,P.Clements,and R.Kazman,Software Architecture in Practice ,2nd ed.,Addison-Wesley,2003 8
- 9. + Software Architecture Evaluation ENASE'18. Madeira, Portugal. There is therefore a need for software architecture evaluation (SAE) approaches to minimize the negative impact of low quality SA on software implementation. This study aims to get insight into the existing SAE approaches 9
- 10. + Method Systematic mapping study ENASE'18. Madeira, Portugal. 10
- 11. + Systematic Mapping Process ENASE'18. Madeira, Portugal. Definition of Mapping Questions Conduct Search Screening of Papers Key wording using Abstracts Data Extraction and Mapping Process Review Scope All papers Relevant Papers Classification Scheme Systematic Map Process Steps Outcomes 11
- 12. + Systematic mapping questions • Which publication channels are the main targets for SAE research?MQ1 • How has the frequency of approaches related to SAE changed over time?MQ2 • What are the research types of SAE studies?MQ3 • Are SAE studies empirically validated?MQ4 • What are the evaluation approaches that were reported in SA research?MQ5 • Were SAE approaches reported in literature based on SQ model?MQ6 • Which quality attributes were used to evaluate SA?MQ7 • What are the models that were used in SAE literature?MQ8 ENASE'18. Madeira, Portugal. 12
- 13. + Search strategy “Software architecture” AND (evaluat* OR measur* OR assess*) AND (technique* OR approach* OR method* OR model* OR framework* OR tool*) ENASE'18. Madeira, Portugal. IEEE Digital Library, ACM Digital Library, Science Direct, SpringerLink and Google scholar. 13
- 14. + Study selection Papers that focus on software design EC1 Papers whose subject was one or many quality characteristic s which were not used for SAE EC2 ENASE'18. Madeira, Portugal. • were limited to the studies that address evaluation, measurement or assessment of the SA in overall or through quality attributes. Inclusion criteria 14
- 15. + Study selection ENASE'18. Madeira, Portugal. ACM Digital Library, IEEE Xplore, ScienceDirect, SpringerLink, Google scholar Search string Title, abstract and keywords Inclusion and Exclusion criteria • With duplicates: 978 • Without duplicates: 217 candidate studies 60 selected studies • ACM: 3 • IEEE: 16 • Google scholar: 24 • ScienceDirect: 9 • SpringerLink: 8 15
- 16. + Results Systematic mapping study ENASE'18. Madeira, Portugal. 16
- 17. + MQ1. Publication source ENASE'18. Madeira, Portugal. 25% 48% 9% 12% 3% 3% Journals Conferences Workshops Technical repports Books Thesis 17
- 18. + MQ2. Publication trend ENASE'18. Madeira, Portugal. 2 1 1 2 1 5 5 4 4 3 4 5 5 3 1 4 4 2 2 1 1 § The interest on SAE began after the publication of technical reports by the Software Engineering Institutes in 1997. § This interest was at it most during the last decade where many researchers have based their research on the outcomes of these technical reports mainly on the Architecture Tradeoff Analysis Method (ATAM). § This interest started to fade since 2014, which indicates that there is a need for novel SAE techniques, particularly with the emerging new technologies such as the IoT and the Big Data 18
- 19. +MQ3,4, 5. Results summary ENASE'18. Madeira, Portugal. 19
- 20. + MQ6. Software Quality models ENASE'18. Madeira, Portugal. No 69% Boehm 3% McCall 6% ISO/IEC 9126 14% ISO/IEC 25010 5% Others 3% 20 § Quality is a measure of excellence or the state of being free from deficiencies or defects. § Quality attributes are system properties that are separated from the functionality of the system. § Implementing quality attributes makes it is easier to differentiate a good system from a bad one. § Attributes are overall factors that affect runtime behavior, system design, and user experience.
- 21. + MQ6. Software Quality models ENASE'18. Madeira, Portugal. 21 21 ISO/IEC 9126 quality model Software Quality in Use Software Product Quality External quality Internal quality ISO/IEC 25010 Software Product Quality Functional suitability Reliability Performance efficiency Usability Portability Compatibility Maintainability Security • Functional Completeness • Functional Correctness • Functional Appropriateness • Maturity • Availability • Fault tolerance • Recoverability • Time-behavior • Resource-utilization • Capacity • Appropriateness recognisability • Learnability • Operability • User error protection • User interface aesthetics • Accessibility • Adaptability • Installability • Replaceability • Co-existence • Interoperability • Modularity • Reusability • Analyzability • Modifiability • Testability • Confidentiality • Integrity • Non-repudiation • Accountability • Authenticity ISO/IEC25010QualityinUse Effectiveness Efficiency Satisfaction Usefulness Trust Pleasure Comfort Freedom from Risk Economic Risk Mitigation Health and Safety Risk Mitigation Environmental Risk Mitigation Context Coverage Context Completeness Flexibility Software quality is defined as the “capability of software product to satisfy stated and implied needs under specified conditions” ISO/IEC 25010:2011
- 22. + MQ7. Quality attributes ENASE'18. Madeira, Portugal. Others 22% Performance 20% Maintainability 15% Reliability 12% Security 10% Usability 8% Portability 6% Functionality 4% Compliance 3% Table 3: Papers which have focus on one quality attributes Quality attrib. Ref. Total Maintainability (Bosch and Bengtsson, 2001) (Bouwers, 2013) (Graaf et al., 2005) (Lindvall et al., 2003) (Tvedt et al., 2002a) 5 Performance (Akinnuwesi et al., 2013) (Knodel and Naab, 2014) (Martens et al., 2011) (Reijonen et al., 2010) 4 Security (Ali Babar, 2008) (Alkussayer and Allen, 2010) 2 Usability (Folmer et al., 2003) 1 Adaptability (Liu and Wang, 2005) 1 Changeability (Subramanian and Chung, 2004) 1 3.7 MQ7. Quality attributes 15% of SAE papers did not mention any quality at- tribute. Fig. 4 shows how often a quality attribute has been mentioned in SAE literature. It is worth men- tioning that some papers dealt with the evaluation of SA through only a unique quality attribute. Table 3 presents these characteristics. 4 DISCUSSION The interest on SAE began after the publication of technical reports by the Software Engineering Insti- tutes in 1997. This interest was at it most during the last decade where many researchers have based their research on the outcomes of these technical reports mainly on the Architecture Tradeoff Analysis Method (ATAM). However, this interest started to fade since 2014, which indicates that there is a need for novel SAE techniques, particularly with the emerging new technologies such as the IoT and the Big Data (Krco et al., 2014; Gorton and Klein, 2015). The majority of the papers were evaluated using case studies, it is eas- ier to evaluate SA of existing systems rather than de- veloping a system only for the purpose to evaluate its architecture. In fact, SA requires an early software en- gineering activity which is the specification of stake- holders needs, also known as requirements engineer- ing (Ouhbi et al., 2013). This step is critical to iden- Papers which have focused on one quality attributes 22
- 23. + MQ8. SA description models ENASE'18. Madeira, Portugal. 4+1 view model 25% ADL 25% UML 50% 23 A software architecture must describe its group of components, their connections, interactions among them and deployment configuration of all components. A software architecture can be defined in many ways: § UML (Unified Modeling Language) § UML is one of object-oriented solutions used in software modeling and design. § Architecture View Model (4+1 view model) § Architecture view model represents the functional and non-functional requirements of software application. § ADL (Architecture Description Language) § ADL defines the software architecture formally and semantically.
- 24. + MQ8. SA description models ENASE'18. Madeira, Portugal. 24 Logical view/ Conceptual view Development view Process view Physical view Scenario view/ Use case view Describes the object model of design (Object-oriented decomposition) Describes the structure of the software in its development environment Describes the activities of the system (Process decomposition) Describes the mapping of the software onto hardware Putting it all together Class, State, Object, Sequence, Communication Diagram Activity Diagram Component, Package diagram Deployment diagram Use case diagram 4+1 View Model
- 25. +Conclusions and Implications ENASE'18. Madeira, Portugal. o The overall goal of this study is to conduct a thematic analysis and identify publication fora as regards SAE approaches. o The findings of this systematic map have implications for researchers and practitioners who work in the SA domain, since this study will allow them to discover the existing SAE approaches and techniques in the literature. o The presented empirical studies may also provide an overview of the efficiency of each approach. Future research o Systematic literature review on empirical SAE studies 25
- 26. + Thank you for your attention! Any questions? ENASE'18. Madeira, Portugal. 26
- 27. + 13th International Conference on Evaluation of Novel Approaches to Software Engineering. Funchal, Madeira, Portugal 23 March, 2018 Presented by: Sofia Ouhbi Software Architecture Evaluation A systematic mapping study Sofia Ouhbi ENASE'18. Madeira, Portugal.