back to notes

Managing Technical Debt

Managing Technical Debt_Virtual Coffee.pptx

Technical debt is a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution. The Technical Debt concept is an effective way to communicate about the need for refactoring and improvement tasks related to the source code and its architecture. If you are able to estimate roughly the time needed for fixing what is not right into your code, the principal of your debt, you could compare this information to other project data, like remaining days before release date. This estimation will help you to understand your situation and plan repayment activities. That is, it implies that restructuring existing code (refactoring) is required as part of the development process. Under this line of thinking refactoring is not only a result of poorly written code, but is also done based on an evolving understanding of a problem and the best way to solve that problem. Technical debt may also be known as design debt.

What incurs Technical Debt?

Code that is not quite right may include many types of issues. These issues may be related to architecture, structure, duplication, test coverage, comments and documentation, potential bugs, complexity, code smells, coding practices and style. All these types of issues incur technical debt because they have a negative impact on productivity.
Technical Debt may emerge during the life of a project. As time progresses you may understand something new about your application domain. You may now view your initial architecture as having acquired technical debt.

How to analyze Technical Debt?

All the Technical Debt items are not the same. In order to understand the situation, in order to select and prioritize refactoring or improvement activities, it is important to analyze them. The following table provide a list of aspects that shall be taken into account.

This list represents the good sense that teams should apply, and is largely self-explanatory. However, it is worth saying a few more words about the last point. The Technical Debt concept is quite simple. When looking into details, it appears that they are many aspects to consider for analyzing and managing the situation.

The concept does not mean that debt should never be incurred. Just as leverage can help a company when used correctly, a quick solution can mean a faster time to market in software development. In addition, technical debt is not just poor code. Bad code is bad code, and technical debt can result from the work of good programmers under unrealistic project constraints.

You have to make that release date. You need more time to get the structure of your modules right, but you don’t have it. Hitting the release date is more important than cleaning your code, so you defer the cleanup to make the deadline. You agree to take on technical debt which you’ll have to pay back later. Speeding up now to hit this release date will make you slower hitting the next one. If you’re not careful, over time, development slows down to a crawl and every feature takes weeks instead of days.

Let’s look at some examples and see what technical debt is, and what it isn’t.

A typical example for incurring technical debt in code is that huge source file every developer fears to touch. Everyone on the team knows that it needs to be broken into more manageable pieces, but it would delay upcoming features by at least two weeks. Touching this module without refactoring it is adding to the technical debt of your product. Every such change will make the eventual refactoring harder and therefore needs to be paid back later – with interest. The interest you’ll have to pay is the added complexity of every change you added. The module grows with every code addition making its refactoring (and even the decision to do so) ever more difficult.

An example for technical debt from the operations side of things is deploying your application in only one availability zone in your datacenter even though you have demanding uptime requirements. Setting up a split environment, which covers multiple availability zones is more work. It might be necessary to defer this additional work to be able to deliver critical features to your clients. In this scenario, technical debt does not materialize in added complexity but in higher risk of downtime. To avoid necessary work right now, you introduce single points of failure which will ultimately lower your uptime.

So, what isn’t technical debt?

Technical debt can be defined as the longer-term consequences of poor design decisions. It's worth bearing in mind, however, that not all flaws and defects constitute technical debt. This is because they don't reflect "design decisions" which were actually taken. They are often just errors, no matter how irresponsible or egregious they might be. Also, if a decision doesn't directly compromise product quality, then it isn't technical debt. Hence a team may wish for a slick new IDE or plug-in, but the failure to provide the same isn't "technical debt", since it doesn't put the quality of the product itself at risk. It might very well reduce velocity, because they have to limp on using the old development platform, but that's a separate concern.

Are there other types of debt?

Not all software projects issues are Technical Debt:
· Identified defects are not Technical Debt. They are Quality Debt.
· Lack of process or poor process is not Technical Debt. It is Process Debt. An example is Configuration Management Debt.
· Wrong or delayed features are not Technical Debt. They are Feature Debt.
· Inconsistent or poor user experience is not Technical Debt. It is User Experience Debt.
· Lack of skills is not Technical Debt. It is Skill Debt.

Is Technical Debt bad?

Taking short cuts in order to put earlier a viable product on the market which delivers business value is generally not a bad decision. But one should be conscious that the Technical Debt incurred will hurt sooner or later.

At some time, the team should pay back at least a part of the accumulated Technical Debt. They are different ways to do that, and there is no magic one that fits all situations. In order to fully understand the situation and establish the relevant strategy, the Technical Debt should be made fully transparent and analyzed.

How to solve for Technical Debt

In Scrum, the expectation is that a Definition of Done should be sufficiently robust for unmanageable levels of genuine debt not to accrue. Hence if technical debt is known to be building up, the Definition of Done should be revisited. It's essential to find out why the debt is being incurred and how this can be avoided.

There are certain other controls which can be used to keep debt down. For example, a team should allow for refactoring (and indeed any other tasks) when deciding how much work they can induct into a Sprint Backlog without compromising long-term quality. Yet apart from these checks and balances, there's no prescription for how technical debt should be handled once you've got it. This isn't an oversight, since Scrum is deliberately as non-prescriptive as possible. It's a framework, and it's up to teams how they implement it.

**Implementing special sprints to clean up technical debt isn't an option. Technical debt sprints, also referred to as hardening sprints, are essentially an anti-pattern. Each and every Sprint must yield an increment of genuine release quality. That's why the Definition of Done is the primary bulwark against debt building up in the first place.**

What some teams do is to maintain a technical debt register, whereby the design decisions which lead to debt can be rationalized. You can think of this register as a RAID log (Risks, Assumptions, Issues, and Dependencies) which is under the team's own purview. It details the technical consequences of expedient decisions on the quality of product implementation, often in terms of probability, impact, and suggested remedy.

A technical debt register can help inform teams about how the debt they incur should be managed. They can make sensible, informed decisions about whether or not to incur debt and when to pay it back. The use of a register like this isn't part of Scrum, but nonetheless it can help a team to get a grip on the technical debt they decide to take on. Sometimes, for example, technical debt can be paid off when implementing related backlog items. In other cases that might not be possible and the debt must be addressed separately.

Certain instances of debt may need to be exposed to the Product Owner for consideration, and others may not. In severe cases technical debt may have been accumulated which exceeds the value of the project itself, possibly by multiple times. In such extreme cases the most pragmatic approach may be to can the project and start again. Needless to say, this variation on "fail now, not later" takes extreme courage.

Another option when faced with substantial technical debt is to whittle it down gradually, Sprint by Sprint. The team may need to conspire with the Product Owner to deliver something, each and every iteration, which releases sufficient value to stakeholders. Clearly this isn't a great option. It can end up in horse-trading, where the scale and ownership of the debt problem is not admitted to or made known to those stakeholders. Technical debt then becomes more a case of technical embezzlement. Affected parties might be encouraged to turn a blind eye as long as they get something of immediate business value in return, but none of this is good for openness, trust, and transparency.

References:
https://www.agilealliance.org/introduction-to-the-technical-debt-concept/
https://www.techopedia.com/definition/27913/technical-debt
https://agileweboperations.com/2015/09/30/technical-debt/
https://www.scrum.org/resources/blog/using-technical-debt-register-scrum