Alessandro DiaferiaThe programming community unanimously considers technical debt an aspect of our work to keep under control and reduce.
Personally, I’ve been vocal about the perils of technical debt in one of my early blog posts about the organizational issues it can cause.
While I still stand by the majority of what I described in that article, I want to clarify my take on what I call the presumed technical debt.
What is technical debt?
First of all, though, let’s agree on what the term means. Technical debt is a broad term that encompasses many aspects of building and delivering software.
Martin Fowler refers to it as the cruft that builds up over time and that makes it harder to modify and extend the system further. He then references the likely original author of the metaphor Technical Debt in the person of Ward Cunningham.
What Ward Cunningham wrote first about technical debt , as follows:
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.
Ward Cunningham, 1992
Here I interpret not-quite-right code as a way of referring to the design of code based on assumptions and understanding before it gets actually validated by the end users. But I’ll clarify this further later.
Finally, I would add my own definition of what technical debt is:
Technical debt is the gap between the current state of the software non-functional characteristics and the desired state they should be in, provided you had infinite time, money and absolute understanding of the problem at hand that your software is meant to solve.
Different forms of technical debt
There is a substantial natural cruft that builds up in software because of the different forces that affect it: dependencies getting stale, requirements or user needs changing, more code being added, unused code paths, and so on.
This type of natural technical debt should be continuously monitored and kept under control. You won’t ever be able to get rid of it completely but there should be a continuous tension to keep it under control.
In my experience, though, there is an additional form of technical debt that is less tangible, while representing a considerable influencing factor when developers design software: the presumed technical debt.
Presumed technical debt is a form of debt that developers think they will incur into soon if certain non-functional characteristics do not get implemented. Let’s explore together this form of technical debt.
Non-negotiable characteristics
First of all I want to clarify that there are surely certain baseline non-negotiable characteristics that the organization might require before any code gets into production. These characteristics are in the form of, for example:
- observability and operational envelopes
- security and compliance requirements
- data retention and governance constraints
But there are other characteristics, like scalability, availability or resilience, that depend heavily on the specific use you are building to solve for.
Technical debt or just lack of aspirational characteristics?
On the contrary, in my experience, it’s not unusual for developers to design new functionality targeting infinite scalability , eternal availability , absolute resilience.
At times it feels that those characteristics are the default starting point. Developers aim at those but then compromise back as they face business constraints like time and money.
The MVP ends up not being shipped on a container-based cloud infrastructure that auto-scales. Turns out, the same could be delivered via a static HTML page hosted on S3 and served via CloudFront.
Nevertheless, many teams will still aim at container-based cloud infrastructure as the next iteration for what they shipped. They add that aspiration to their backlog, regardless of the feedback they receive from the MVP.
It’s not unusual to think that what they have shipped is not fit-for-success.
When success comes, it’ll bring troubles to this architecture
Anonymous developer
Having compromised on some ideal characteristics, some developers might not feel happy about the MVP. They might think it won’t certainly scale when success brings troubles. They will consider the current architecture as technical debt, necessarily, and aim for getting rid of it.
I would argue, instead, that those characteristics are aspirational and need to be validated. I’ll tell you more: their lack thereof might never turn into debt, effectively. This is especially true when working on new functionality.
This is why I call it Presumed Technical Debt. It’s coming from the non-validated expectation that the software will definitely need those non-functional characteristics to deliver value effectively.
In this case the technical debt is represented by the gap between the current state of the non-functional characteristics and the ideal version of them as desired by the developer.
Non-validated assumptions are a form of technical debt
Say the Product Organization has identified a new customer problem to solve. In an agile context you would work with your team to identify the minimum viable iteration that your customer could consume so that you can, to name a few:
- validate your understanding of the problem
- understand how your customer would use your software to solve the problem
- understand what is critical minimum functionality for your customer
- understand traffic patterns, auditing requirements, scalability needs, and so on
You are making a bet on what you think is the right way of solving a problem and you want to validate this assumption early.
You have an understanding based on your experience and ideas and you need to get something out there quickly before you go down a path that might take you far away from the real customer needs.
Similarly, you might have assumptions regarding what the non-functional characteristic of your software should look like: does it really needs three nines of minimum availability? The reality might be that no one is expecting any traffic during weekends so an 80% SLA is more than enough!
There is a great article Kurt Bittner and Pierre Pureur about continuously balancing a Minimum Viable Architecture with the Minimum Viable Product.
Investing in more than is needed to support the necessary level of success of the MVP would be wasteful since that level of investment (i.e. solving those problems) may never be needed; if the MVP fails, no more investment is needed in the MVA.
Kurt Bittner and Pierre Pureur – Agility and Architecture: Balancing Minimum Viable Product and Minimum Viable Architecture
Building expensive non-functional characteristics is a form of debt that you might have to carry throughout the whole lifecycle of the software you are building.
If you start with infinite horizontal scalability you are deliberately adding cost, overhead and constraints to keep up with this non-functional characteristic that you haven’t validated yet.
Don’t be DRY at all costs
Another common fallacy I see is the application of the DRY principle as an end goal. DRY stands for Don’t Repeat Yourself and it is often the reason why incredibly complicated abstractions exist.
I attribute many erroneous implementation of the DRY principle to what I call Presumed Technical Debt.
It’s not rare for developers to deem two use cases similar enough that they require a common abstraction implementation. In my experience it’s unsurprisingly frequent to end up with an abstraction that hardly solves either use cases.
Often, developers build the abstraction out of the conviction that repeated code is a form of technical debt to avoid.
In my experience the reality is that by the time you come up with the idea that two use cases might be similar enough to require a common abstraction, you actually don’t have enough data points to be able to make that decision.
Therefore, developing an abstraction out of a non-validated idea regarding commonality among two or more use cases results in an artificial constraint. Such constraint will impact your pace and your ability to iterate swiftly as you receive feedback signals from what you have put in the hands of the customers.
Only then you will actually have data to be able to decide whether you need an abstraction or not. Maybe you need a third use case first!
There is a great talk by Dan Abramov regarding The WET Codebase that highlights the dangers of always striving for abstracting out code too early.
Presumed technical debt makes your job frustrating
Building software chasing an aspirational architecture is a recipe for frustration.
This is why one should not use the term “technical debt” lightly. Labeling the lack of an arbitrarily ideal set of non-functional characteristics as technical debt will always create tension towards achieving an ideal architecture, regardless of whether it is necessary or not.
In my experience the reality is different: most of the times the functionality you have built will need far-from-ideal non-functional characteristics: as developers, sometimes we hope we are going to be given the chance to work on complex architectures , so we start with that upfront.
A truly agile approach would leave many of the made up non-functional requirements to be identified later, as the functionality starts getting used. Will it actually need to scale that much or is it rather delivering great value to my users while being interacted with sporadically? Will my users ever notice the system becoming unavailable at weekends?
As professionals it is our job to build for sustainability. If we design arbitrary, non-validated, non-functional characteristics we are actually building technical debt into our software unnecessarily. An overly complex architecture that doesn’t serve any valuable purpose for the end user might just add development overhead and slow down your organization.
Closing thoughts
In this post I have highlighted what I find being an emerging trend in software engineering: designing non-functional characteristics upfront, based on non-validated assumptions, hoping to be preparing software for success and avoiding technical debt.
In reality, non-validated non-functional characteristics can turn into technical debt if they don’t end up supporting the real needs of the customers.
While it is important to acknowledge the existence and impact of technical debt, it is equally crucial to differentiate between necessary technical debt and idealized technical debt.
If baseline non-negotiable characteristics such as observability, security, and compliance requirements must be met, it is essential to validate and prioritize other characteristics such as scalability, availability, and resilience based on customer needs and feedback.
Developers often fall into the trap of aiming for an idealized architecture before fully understanding the problem and user requirements. This leads to the development of code and architectures that may not align with the actual needs of the software. Making assumptions and designing for non-validated technical requirements can introduce unnecessary complexity and hinder the agility and speed of the development process.
Rather than striving for an abstract ideal, it is important to approach software development with a mindset of sustainability and continuous reassessment. This means continuously monitoring and addressing technical debt that naturally accumulates over time, while avoiding the introduction of additional debt through arbitrary and non-validated technical requirements.
I hope you enjoyed this post. If you like my writing you can follow me on LinkedIn to stay up-to-date on my thoughts on software engineering.
Photo by Martijn Baudoin on Unsplash
The post Presumed technical debt: how to recognise it and avoid it appeared first on Alessandro Diaferia.
