How can you use metrics to identify and mitigate technical debt in system quality attributes?

Technical debt is the cost of compromising system quality attributes, such as performance, security, reliability, or maintainability, for short-term benefits, such as faster delivery, lower cost, or easier development. Technical debt can accumulate over time and negatively affect the system's functionality, usability, scalability, and resilience. To identify and mitigate technical debt, you need to measure and monitor the system's quality attributes and the factors that influence them. In this article, you will learn how to use metrics to assess and improve system quality attributes and reduce technical debt.

1 What are system quality attributes and metrics?

System quality attributes are the non-functional requirements that define how well the system performs its functions and meets the expectations of the stakeholders. They are also called -ilities, such as availability, usability, modifiability, etc. Metrics are quantifiable indicators that measure the degree to which the system satisfies its quality attributes. Metrics can be collected from different sources, such as code analysis, testing, monitoring, feedback, or surveys. Metrics can help you evaluate the current state of the system, identify potential issues, and track the progress of improvement actions.

Add your perspective

J Harlow

Co-founder, executive, mentor, inventor, investigative jounalist.
Report contribution
There are three types of software metrics in software engineering: product metrics, process metrics, and project metrics. Measuring scalars (i.e. packet/sec) in contradistinction to point-values such as “usability” (scale of 1 to 10) must be clearly understood; quantitative-vs-qualitative measures. Scalars often have multi-variate influences requiring documentation of dependencies (network availability, jitter, processor-core(s) used, L2 & L3 caching, and number of green-threads/P-thread, etc). Whatever stakeholder wants, must be delivered, and if they made a mistake in their measures, it’s for engineers to point it out in context of “cost to the business if ignored”. We’re all on the same team - when stakeholder wins, engineers win.

Like

2 How to choose relevant metrics for system quality attributes?

When selecting metrics for system quality attributes, there is no single solution that works for everyone. Instead, metrics should be chosen based on the system's context, goals, and constraints. It is also important to consider the trade-offs and dependencies between different quality attributes, as improving one can have an effect on another. For example, increasing security may reduce performance or usability. When selecting metrics, you should ensure they align with the system's objectives and stakeholder's needs, are clearly and unambiguously defined, measurable and verifiable, and balanced between leading and lagging indicators, as well as input and output indicators. Leading indicators are predictive and proactive such as code quality or test coverage, while lagging indicators are reactive and retrospective such as defect rate or downtime. Input indicators measure the effort and resources invested in the system such as cost or time, while output indicators measure the results and outcomes of the system such as customer satisfaction or revenue.

Add your perspective

Yelena Liman

Director of Product | Driving Business Growth through Product Strategy & Customer-Centric Innovation | Scaling SaaS Execution & Cross-Functional Leadership | Ex-Amazon, Ex-Intel
Report contribution
I would look at this question from a business perspective. It's important to choose metrics that can help the leaders make strategic decisions, metrics that enable effective prioritization, resource allocation, and alignment with business objectives. For me, it's about optimizing efficiency and ensuring technology investments directly contribute to the bottom line and long-term business success.

Like
J Harlow

Co-founder, executive, mentor, inventor, investigative jounalist.
Report contribution
Instrumenting a “production system” can be fraught with headaches especially if using 3rd party tools. The customer will rely upon engineers to exercise workman-like competence and make judgement calls as to “what do we measure and make immediately available to stakeholders?” and “what do we set-aside for analytical deep-dives?”. Sometimes a customer may not know - that’s OK - they rely upon engineers and architects to make ‘judgement calls’ to balance all variables- after all, that’s what engineers do. This is only magnified when supporting a plurality of Operating Systems, message buses, IPC protocols and threading models. Metrics in Unix are different from z/OS and iOS. Thus, metrics must be translatable and “normalized”.

Like

3 How to collect and analyze metrics for system quality attributes?

Once you have selected the metrics for your system quality attributes, you need to collect and analyze them regularly and systematically. Automation of the collection and analysis process is key, as well as ensuring the accuracy and reliability of the data sources and methods. Establishing baselines and benchmarks, identifying trends and patterns over time, and using statistical methods are all important for interpreting and validating metrics. Various tools and methods can be used to gather and process metrics, such as code analysis tools, testing tools, monitoring tools, feedback tools, or surveys. Dashboards, reports, or visualizations can be utilized to display and communicate metrics.

Add your perspective

J Harlow

Co-founder, executive, mentor, inventor, investigative jounalist.
Report contribution
Context is in order. In businesses which subscribe to regimented “Dev”, “Test”, and “Prod” (isolated Development, isolated test & live-firewalled production environments) metric capture is shut-off, generally, once a system is put into production. Sensors can represent cyber-security risks, so, compiled production code will be devoid of nearly all sensors. At best, a minimum set of production metrics are captured from off-line logging which do not adversely affect “production” software. Logging is vital in “prod environments” for forensic purposes and system debugging, but not much more. System-owners provide constraints, engineers meet, or exceed, constraints in Dev & Test. Metric constraints should already be met by time of “Prod”.

Like

4 How to use metrics to identify technical debt in system quality attributes?

Metrics can help you identify technical debt in system quality attributes by highlighting discrepancies between the expected and actual performance of the system. For instance, you can compare metrics with predefined thresholds or standards to detect performance debt, compare metrics with historical or industry averages or best practices to detect maintainability debt, and compare metrics with other metrics or quality attributes to identify security debt. For example, if the response time exceeds the acceptable limit, the code complexity is higher than the average or recommended level, or the security metrics are low while the functionality metrics are high, it may indicate a technical debt.

Add your perspective

5 How to use metrics to mitigate technical debt in system quality attributes?

Metrics can help you mitigate technical debt in system quality attributes by guiding and prioritizing the improvement actions and evaluating their impact and effectiveness. To reduce technical debt, you can set goals and targets for improving metrics and quality attributes, plan and implement improvement actions based on metrics and quality attributes, as well as monitor and review the results of improvement actions using metrics and quality attributes. For example, if you want to reduce performance debt, you can set a goal to decrease the response time by a certain percentage or amount. Additionally, if you want to reduce maintainability debt, you can refactor code, conduct code reviews, or create documentation. Finally, if you want to reduce security debt, you can track changes in security metrics such as vulnerability, risk, or compliance.

Add your perspective

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

Add your perspective

How can you use metrics to identify and mitigate technical debt in system quality attributes?

1

2

3

4

5

6

1 What are system quality attributes and metrics?

2 How to choose relevant metrics for system quality attributes?

3 How to collect and analyze metrics for system quality attributes?

4 How to use metrics to identify technical debt in system quality attributes?

5 How to use metrics to mitigate technical debt in system quality attributes?

6 Here’s what else to consider

System Architecture

Rate this article

Thanks for your feedback

More articles on System Architecture

More relevant reading

How can you use metrics to identify and mitigate technical debt in system quality attributes?

1

2

3

4

5

6

1 What are system quality attributes and metrics?

2 How to choose relevant metrics for system quality attributes?

3 How to collect and analyze metrics for system quality attributes?

4 How to use metrics to identify technical debt in system quality attributes?

5 How to use metrics to mitigate technical debt in system quality attributes?

6 Here’s what else to consider

System Architecture

Rate this article

Thanks for your feedback

Explore Other Skills