How to Move a Large Codebase to a New Stack: A Developer’s Dilemma

As the head of engineering at a startup, I’ve witnessed firsthand the evolution of our technology stack. My journey began as a front-end developer, eventually transitioning to full-stack development, and ultimately leading me to my current role. This trajectory has granted me a unique perspective on our tech decisions, especially those made early on, which I now see as questionable at best.

The Challenges of Our Current Stack

One of the most significant challenges we face is our backend, which is built on a headless CMS called Payload CMS. While Payload CMS is indeed a fantastic tool, using it as the foundation for our entire backend has its downsides. Additionally, we’ve opted for MongoDB, a choice that has led to numerous complications without providing any tangible benefits. To be frank, I believe MongoDB has caused more problems than it has solved for us.

The crux of the issue is that our current stack hampers our developer experience. Tasks that should take a mere 20 minutes using Prisma with a relational database can drag on for an entire day in our existing setup. The thought of transitioning to a relational database, such as PostgreSQL, feels like a dream compared to the frustrations we currently endure.

The Dilemma: Can We Transition?

Despite the glaring issues, the prospect of moving off our current stack feels daunting. We have thousands of references to Mongoose, and we rely heavily on Atlas Search, which works well for us. Our application is complex; it’s well-tested but also intricate, making any significant changes risky and potentially disruptive.

This raises the ultimate question: Is there any hope for us to move off this stack, or is it merely a pipe dream? The thought of being forever shackled to a codebase that is more of a headache than a help is disheartening. However, I wonder if there’s a feasible path to improvement.

Understanding Stakeholder Perspectives

One of the most significant obstacles to transitioning our stack is the perspective of business stakeholders. From their viewpoint, investing time and money into a project that doesn’t seem to add value to the product can be a hard sell. Improvements to developer experience, while crucial for long-term sustainability and productivity, may not resonate with those focused on immediate business outcomes.

To make a compelling case for a transition, we must articulate the long-term benefits clearly. For instance, improved developer experience can lead to faster feature development, reduced bugs, and ultimately a better product for our users. Framing the discussion around these points may help garner the necessary support for a transition.

Potential Strategies for Transition

While a complete overhaul of our codebase might seem daunting, several strategies could facilitate a smoother transition:

1. Refactor, Don’t Rebuild: Instead of a complete rewrite, consider gradually refactoring parts of the codebase. Identify the most problematic areas and tackle them incrementally. This approach allows for continuous delivery of features and improvements while minimizing disruption.

2. Strangler Pattern: This architectural pattern allows for a gradual replacement of parts of the system. By creating new services that replicate the functionality of the existing system, you can slowly phase out the old components. This method ensures that you can maintain and improve the system without facing a complete shutdown.

3. Incremental Migration: Start migrating specific features or services to a new stack while keeping the rest of the application intact. This approach allows you to test and iterate on the new stack without risking the stability of the entire application.

4. Utilize the Existing Infrastructure: Leverage tools and services that integrate well with both MongoDB and your desired relational database. For instance, consider hybrid solutions that allow for gradual data migration while maintaining functionality.

Conclusion: A Path Forward

While the challenges of transitioning to a new stack are significant, they are not insurmountable. By focusing on incremental improvements, leveraging the strangler pattern, and making a compelling business case, we can pave the way for a better developer experience and ultimately a more robust product.

The journey may be difficult, but with the right strategies in place, we can transform our codebase into something that not only functions well but also serves our team’s needs more effectively. Here’s to hoping for a brighter, more manageable future in our development endeavors!