Reduce Tech Debt + Prevent New Tech Debt with These DevOps Tips

Tech debt affects just about every software organization. For better or worse, speed is one of the core tenets of the software development industry – the rush to build more, add new features, and find solutions fast can drive short-term growth for small organizations and add value for large companies. But haste, as it usually does, makes waste: The mass of suboptimal code, redundant services, and bottlenecked processes that we call tech debt.

You’re probably affected by some amount of tech debt in your daily responsibilities, whether you’re a developer, IT ops professional, or even a user of technology produced by those teams. Let’s take a look at what tech debt does to a software development team, how it happens, best practices for reducing tech debt now, and what you can do to reduce tech debt in the future.

What is Technical Debt?

Tech debt is the gradual accumulation of low-quality code, redundant services, and bottlenecked processes in software development. Tech debt is often caused by rushed work and tight deadlines. Shortcuts in coding might improve your velocity in the short term, but they require costly maintenance and fixes later on.

The consequences of tech debt can range widely, from decreased efficiency all the way to security vulnerabilities. Tech debt can introduce bugs, slow down users, impact system reliability, make it hard to scale up and accommodate more traffic/usage, and be a drag on team morale (which makes all those other problems worse).

How Tech Debt Happens

Developers are incentivized to build solutions that will meet business objectives on time, even if it’s not made for the long term. When it achieves the business objective, it usually gets forgotten and built onto – until it becomes a problem. But because software development moves too quickly for thorough review and refactoring, it’s easy for tech debt to pile up fast without being noticed.

A few common causes of tech debt include:

• Tight deadlines that lead to coding shortcuts
• High expectations that necessitate short-term solutions
• Scope creep that forces quick solutions to meet changing project requirements
• Lack of code testing or a strong CI/CD pipeline
• Building on old code instead of improving or modernizing it
• ‘Band-aid’ quick fixes that don’t align to best practices for maintainability
• Skill gaps that leave new developers stuck with code they didn’t write and don’t understand

Worse yet, every new instance of tech debt makes it harder to reduce the tech debt you already face. To reduce tech debt, your team would need to spend valuable time cleaning up all that old, suboptimal code instead of writing new code that contributes to your organization’s profit-driven bottom line. You can see how that would add up to a bit of a vicious cycle.

Technical Debt and DevOps

Tech debt can be a huge DevOps blocker. Tech debt leads to reduced agility, higher fail rates, difficulty scaling DevOps, decreased security and compliance posture, and more.

Technical debt slows down teams, decreases productivity, and creates bottlenecks. Those lead to team-wide inefficiencies, security risks, compliance errors, and a terrible developer experience. It affects many levels of a DevOps organization, from developers and DevOps engineers all the way up to business leaders.

In a DevOps context, you could say that tech debt makes DevOps harder to do – and conversely, DevOps can be one of your greatest assets in kicking tech debt. The core tenets of DevOps are focused on the kinds of problems that lead to tech debt, like poor planning, inadequate testing, scope creep, and duplication of work.

A Guide to Tech Debt Reduction: Finding + Fixing Tech Debt Across Teams

By the time you notice tech debt, there’s not really time to uproot everything and fix it then and there. Even if there was time, it’d be way too risky to roll out code changes to your underlying infrastructure willy-nilly. (That’s how you got all that tech debt in the first place, remember?)

No, that won’t work. If you want to reduce tech debt in your organization, you’ll have to get disciplined and proactive about it.

Finding Tech Debt

The hard part about tech debt is that it slowly becomes normalized, which leads to it becoming part of the status quo, sort of like background noise. In fact, your developers probably stumble over it all the time in the course of regular software development – but they accept that it’s just ‘the way we do it’ or a ‘necessary’ part of the process.

The best way to surface the most painful tech debt is to conduct code reviews. Be on the lookout for telltale signs of tech debt:

• Code smells like dead code, function names that no longer reflect what they actually do, classes with too many responsibilities or methods, and more
• Overly complex code like unnecessarily complex structures and deeply nested conditional statements
• Duplicate code or code that looks like it was copied and pasted across the codebase
  • Extract them into reusable functions or components instead!
• Noncompliant code that doesn’t adhere to your organization’s coding guidelines, formatting, or best practices
• Old dependencies that reference unsupported or vulnerable components

For example, not long ago I ran into a function named capitalize(str). Except it didn’t capitalize the string. It actually lowercased it, replaced some characters with underscores, removed other characters, and various other transforms. It was complex and had strange nested conditionals.

After a bit of spelunking, I determined that it was actually normalizing a string into a GitHub username. My best guess was that it had been repurposed and then evolved over enough time that it no longer resembled what it started as!  I didn’t have time to unwind the conditionals and simplify the strange logic, but I renamed the function to normalizeToGithub(username) and now it was clear what it was being used for (and it will be far easier to search the codebase next time we want to refactor!)

Once you know where tech debt hides, take the appropriate action to correct it. Refactor code, document changes, and share what you find with the rest of the development team.

Fixing Tech Debt: Reduction Tools + Strategies

Obviously, the best way to fix tech debt is to avoid falling into that trap altogether. But it’s a slippery slope, and one that’s easy to find yourself at the bottom of. It turns out that the principles of DevOps can help you fix and prevent tech debt.

• Plan and prioritize. Reducing tech debt is going to take a lot of coding, refactoring, and testing. Don’t go in without a roadmap that outlines your goals, timelines, and resources – all the way from investigation through remediation and setting new best practices.
• Assess your current state. Take a look at your codebase, infrastructure, and documentation to find out where you’re indebted to suboptimal code and software. That’ll help you identify the areas and ways in which tech debt is hitting you the hardest.
• Document everything and maintain that documentation. A shocking amount of tech debt occurs because the developer simply didn’t know that a problem had already been solved, or that there was a better way to do something. I’ve worked in codebases that had four or five “same-but-different" functions to do the same thing. Whether it’s making more descriptive function names or maintaining an architecture doc showing how to navigate the codebase, you should always aim for discoverability and clarity.
• Refactor in small steps. Rather than an all-at-once approach that’s likely to fail, iterate in small and testable discrete steps. Take the capitalize() function above. Rather than fix everything, I just clarified the function’s purpose. Complete CI testing caught any uses of that function that I forgot to update. Dedicate a few hours each week to tech debt reduction.
• Automate your CI/CD pipeline. A CI/CD pipeline powered by strong automation can enforce consistent processes from development through deployment – and catch errors, suboptimal code, or code that falls outside your standards and best practices before it gets committed. That way, you’re identifying tech debt issues as they pop up, rather than having to do massive reviews and refactoring all the time.
  • If you don’t have a CI/CD pipeline in place, build one. If you DO have CI/CD pipeline, it should regularly help you uncover tech debt—or prevent you from committing it in the first place. Learn more about putting together a CI/CD pipeline >>
• Use Infrastructure as code (IaC) to keep things consistent. If CI/CD can make sure code is clean before it goes out the door, IaC makes sure that infrastructure setups are consistent. IaC leverages configuration states that it can reference every single time a resource gets spun up, ensuring all infrastructure adheres to your organization’s policies and best practices.
  • Remember: IaC is the code itself, and policy as code (PaC) is a guideline you can set for using that code. Using PaC enforces repeatable configurations specific to your IT needs. Both are great for defining and enforcing your optimal infrastructure configurations. Find out more about PaC vs. IaC >>
• Use monitoring and observability tools to nip potential tech debt in the bud. Tech debt gets harder to address the longer it sticks around. Observability and monitoring tools give you insight into system performance and stability metrics, which can help identify and prioritize the red flags of tech debt before it becomes a hassle.
• Collaborate, even with teams that don’t do coding. Even though developers will be the closest to the actual sources of tech debt, working across teams can help you find and tackle it. Set up meetings and carve out time in a sprint or cycle for retrospectives. Foster knowledge sharing between dev, ops, and other teams (like SecOps, your infrastructure team if it’s separate, and the platform engineering team). Collaboration helps get everyone on the same page about what tech debt is, how it affects different teams, and what you can do as a group to address it.

How to Keep Tech Debt at Bay

The best way to prevent tech debt is to set up regular code reviews, automate testing and scanning, and use infrastructure as code. Those tactics will help reveal tech debt before it becomes a burden and make it easier to handle in the future.

In a perfect world, you’d have the time to go back and fix all the old code that’s leaving you stuck with a bunch of tech debt. Congratulations! Everything’s better now. Except… now you’ve got to make sure it doesn’t happen again!

In fact, reducing tech debt is only the first step toward keeping code clean. Tech debt prevention might seem like a pipe dream, but it’s possible if you’re able to commit to it.

• Set up smart automations that meet your organization’s best practices. Automate testing and security scans to catch tech debt before it piles up. Remember: No cutting corners! Automation is only good when it sets up safe shortcuts.
• Align configurations to industry standards like CIS Benchmarks, DISA STIGs, and whatever else applies to your industry or service-level agreements (SLAs).
• Set up a CI/CD pipeline (or strengthen the one you’ve got).
• Make time for regular refactoring and code improvement. Involve as many departments as makes sense for your organization.
• Before writing something new, dig deep to see if it already exists. Reuse code when it makes sense and refactor when possible to make code reusable.
• Design component boundaries with consistent, stable, and documented interfaces. Rather than deep linking into private methods and creating tight and fragile coupling, plan for reuse from the beginning. This lets you fearlessly refactor the internals of individual components as long as you maintain the integrity of the interface.

It’s important to remember that for most organizations, tech debt is inescapable. In fact, it’s often accumulated intentionally the same way you might take out a loan to purchase a house or car today instead of saving for years and then paying a mountain of cash. But while you might not be able to prevent a bit of cruft from getting through here and there, you can manage and mitigate it before it becomes insurmountable. Keep tech debt on a short leash and you can prevent it from hampering more efficient software development.

Curious how Puppet fits into all this? Puppet’s industry-leading automation is built to make complex infrastructure manageable at scale. Puppet’s agent-based automation works even during network outages, and writing your configurations as Puppet code lets you repeat the same configurations across on-prem, cloud, and hybrid IT.

Check it out for yourself with a free trial of Puppet or request a demo with the Puppet team to see how it helps organizations build, maintain, and manage better systems.

PUPPET TRIAL   DEMO PUPPET