Technical debt - Part 2 - What to look out for? How to work around it in agile and scrum?

What to look out for to detect technical debt in agile or waterfall development?

Right off the bat, it's important to note that discovering that your problem is technical debt may not always be easy. Complex projects being built today can have a variety of issues ranging from inadequate hardware, to poor architecture, to complex and inefficient algorithms, and more.

Second, if you are a person who doesn't understand IT technology, most of the technical information around technical debt will very likely not make much sense to you.

If we go back to our first piece on technical debt, then you already know what can actually be considered technical debt, and that technical debt is more or less a part of every IT project today. Sometimes technical debt is a pre-calculated bet with all the more potential profit. Sometimes, on the other hand, it is just the ignorance of the human mind or the recklessness of the development team. Last case it is "just" the incredibly fast growth of the whole IT industry and so it is possible that what is modern and functional today may be surpassed by another and better technology in two years and then it is up to everyone's judgement whether such technology would fit your project or not.

The bottom line is that you need to focus on a few basic points and by following them, you will be in control in most cases. Essentially, then, it all comes down to communication and transparency of development. Does the development team communicate with you about the status of your project? Do they openly discuss with you the possibilities of developing additional features, and if the chosen solution option might increase the technical debt, do they inform you about it? Is there a major or minor upgrade being done here and there (usually a couple of times a year) to the technologies used in a sprint? Do you have access to the source code?

If the development team is transparent, doesn't lie about the status of the project and explains everything clearly, and also notifies you in advance and asks for your consent in case of a possible increase in technical debt, then everything is probably fine. For example, if you are more technically inclined, you can also have an overview of the technologies used and their versions in one place. But if nothing like that is happening, then it's a good idea to ask the development team a few questions once in a while and get their answers.

Examples of such questions are:

1. How are we doing on technical debt? Is there anything that can affect us negatively in the long term?
2. What are the versions of our libraries and technologies and how far behind are we compared to current major releases? Is there anything in the new versions that can help us?
3. What are our most pressing technical debt issues right now and how does that affect us?
4. Is there anything holding the development team back from effectively developing new features? What can we do about it?

This is just a basic set of miscellaneous questions, however it will often lead you to follow-up questions and answers.

Within this chapter, a small note on using the latest versions of libraries and technologies. The general experience in IT is that unless your project absolutely needs it, or the pros of a new version of a technology outweigh the cons (for example, a large increase in speed and number of requests processed), it is always worth waiting at least a few months after the release of a new major version of a technology. After the release of a new version, there are often minor or major bugs that are still associated with it or that are discovered by so-called early adopters, which are individuals or companies that update to a new version because they have to. These bugs are then logged very quickly, fixed, and after a few weeks or months there is essentially a very stable version that you can work with without any problems.

How to work with technical debt in agile and scrum?

We have already outlined this passage lightly in our first piece on technical debt, but we will repeat it here.

1. Keep in mind that technical debt applies to every project. Sometimes it is just a premeditated debt with the vision of profit, sometimes the situation is worse/different. In reality, it means setting aside the time and therefore the funds to address it.

2. Keep a record of exactly what it is, try to erase the technical debt part by part. A single page in Confluence, an open ticket in GitHub, or a task in JIRA that your product manager has under his/her thumb is enough to keep a record.

3. If you need to do rapid development even at the cost of increasing technical debt to reach a milestone, expect to have to find time in the future to tackle that newly created technical debt.

4. If it makes sense for you, bring in a third-party expert who can perform a technical audit on your project.

5. Don't leave technical debt to fester. The longer it goes unattended, the worse the long-term consequences for the project can be. Prioritize.

6. Have processes in place that can significantly mitigate technical debt. This can be, for example, automated tests, rules for developing new modules or components, stricter code-reviews, etc.

7. Communication between the development team and you as a client is also important. So talk openly about the issue and you are on the best way to always stay one step ahead.

A few examples of technical debt that we have seen in our years of experience

Back in part one of our short series on technical debt, we showed you one example of such an issue and its impact on a project. Today we have three more different examples that we have seen (out of many) since Moravio has been on the market in 2011.

Bad database design

This situation happened in the context of a project we took under our wing at Moravio. When we took over the project, we discovered that the project was poorly designed in terms of database. One main object that was created during a certain action on the portal, while grouping several entities under it, always created its own copy of the database structure in which it stored data. This made the entire database very quickly become cluttered and, in addition, the poor naming of the tables and associated entities made it very difficult to track anything down. This architecture also had a direct impact on the development of some features that could not be implemented by default. With the rapid growth of the project, it was necessary to modify the database and unify everything. This resulted in speeding up the work of the development team, refactoring the part of the code that interacts with the database and also increasing the clarity of the whole database schema.

Neglecting to update technology

This is one of the very common reasons why projects have technical debt, whether it’s agile or waterfall development and this example is given on a project where we delivered our services as part of Co-development The project uses PHP as the main technology. This is a very capable language today that can do defacto everything that any other modern and advanced programming languages (JavaScipt, .NET Core etc.) can do. However, major versions of PHP often contain a large list of changes and also a lot of work directly on the core of the language. The client made two major mistakes in this project from our point of view. The first one is that he delayed - for a very long time - update to a newer technology, because in his case it was not "just" a simple update, but he also needed to refactor part of the code, for which he needed to plan more time. And the second is that once the client had started the update and worked on it for some time, he eventually put it on ice again because other and bigger priorities emerged. In the meantime, the whole project grew significantly because of new features for customers and also another new major version of PHP was released. This resulted in the update being worked on in leaps and bounds and the developers lost track. This made the work very inefficient. However, the benefits of updating to the new technology were significant for the client - a huge increase in application speed, processing more customer requests in the same amount of time, and also an increase in application security.

Not creating automated tests or not testing at all

Today, you will still find many projects where automated tests are not created by the development team. Yes, you can argue that such automated tests cost time and money and you can't really tell, but that would be a slightly naive way of looking at it. Automated tests exist, among other things, to replace human brains and give your project (and developers) the confidence and stability it needs. Because from a certain size of your project, it is no longer possible for our brains to hold all the connections and all the related scenarios of your project's behavior. Automated tests are then created for these cases, which always test the same specific functionality every time you deploy new functionality. Let me give a very simple fictional example:

You use third-party login in your project. These are Google, Microsoft, Facebook and Twitter. Due to an API update on the Facebook side, you need to modify the Facebook login. You make the modification, deploy to a test environment, test, deploy to production, everything works. But in the modification, the source code used by Google, Microsoft and Twitter login providers was also modified. If there are no automated tests in your project, then you'll most likely only find out about this problem when tickets start flying on your customer service line and phones start ringing from angry users who can't use your app and you're dealing with a hotfix. However, if there were properly written automated tests, it would already be discovered when deployed to a test environment that the login feature through Google, Microsoft and Twitter failed the test, so the developer not only looks at the feature they actually modified (Facebook), but also modifies the feature that other login providers (Google, Microsoft, Twitter) are working with.

So yes, automated tests do cost extra time and money and they don't solve all the bugs in a project (bugs are part of every project, just like technical debt) but in the long run, it will save you multiples of that time and money. Mainly because the entire development team can focus on developing new features and not spend time constantly fixing bugs. I've personally seen cases where the development team spends basically half of their entire capacity just fixing bugs, and that's something you don't want on your project.

This concludes our two-part series on technical debt and we wish you to encounter as few examples of the above mentioned problems as possible.

If you need an audit of your current solution, or a general takeover of your current project, get in touch and together we'll figure out the next steps.