Microservices Migration: Lessons Learned

Background

“Microservices” is a buzzword that has been going around the tech space in recent times. We ultimately decided to migrate to a microservices architecture and want to share why we did it, how we did it, and what we learned along the way.Like most web applications, Andela’s systems started out as a “monolithic” software architecture with a couple of web applications, each with its own database and responsible for a business function. For instance, we built different web apps to manage recruiting of applicants, selection, assessing learning performance, and mastery of skills, which are all crucial to developing world-class developers.As our suite of applications grew, we hit three major challenges:

• There was a lot of replicated business logic across apps.
• Connecting data across applications was problematic.
• Growing monolithic codebases resulting in slower product delivery.

Ultimately, we were building standalone monolithic apps to address specific business needs as opposed to an ecosystem of data-connected applications which is more aligned with the way our business operates. It became evidently clear that in order to scale, we needed to rethink our software architecture. In researching different types of architectures, we came across this blog post from nginx, which pushed us in the direction of microservices. After building a case to the company as to why microservices would help solve our challenges, we decided to migrate and embark on the journey.

Execution

We started by learning as much about microservices as possible: what they entail, the challenges they pose, and the impact they could have on our team structure and agile processes. We appointed Ikem Okonkwo to spearhead this research and lead an initial Core team of five developers to get us started. The results of his research and the design considerations made are described comprehensively in this blog post. Once we were comfortable with our findings, we put a plan together and hit the ground running.The core team started by setting up our new infrastructure, creating microservices templates in different languages, setting up our hosting service (we moved away from Heroku to Google Cloud), setting up Kafka (distributed streaming platform), setting up how we manage our servers and clusters (Kubernetes), and setting up continuous integration (ConcourseCI).Once our minimum infrastructural needs were in place, all the feature teams began migrating our products to microservices.

Outcome

Over the course of six months, we decoupled five critical monolithic systems into 35 microservices with front-end apps interacting with the backend systems via an API Gateway. This effort involved 15 engineers working really hard in order to improve our platform.

Lessons Learned

Completing the migration was no walk in the park. We made some mistakes along the way, though fortunately, we were able to learn from them. Here are a few things we wish we had thought about before beginning the migration:

• Learning curve: the mental transition from building apps as a single unit versus smaller, independent, and distributed units. This is important to be aware of because building distributed systems affects teams structure, and how they communicate to get work done.
• UUIDs: Integer IDs with auto-increment work well enough with standalone monoliths, but in a distributed architecture we’d end up running into conflicts. UUIDs allowed us to track the different business logic and connect them across the ecosystem.
• Starter Kits: boilerplate code containing everything necessary to develop and deploy a new microservice. Starter kits were particularly helpful with speeding up development and onboarding of new developers.
• Automation: a continuous delivery pipeline and an automated environment setup enabled us to empower our engineers to write, test and ship code with a high degree of autonomy.
• Finish fast: as we migrated the platform, there was a constant race against feature development on the existing codebase. Migration needed to move faster than the rest of the codebase. It is imperative to have a strong team of engineers that can finish the migration swiftly and quickly.

Despite the challenges we faced, the migration was a success and we believe we made the right decision. We will continue to leverage our learnings to improve our platform and software development processes. We are now more than ever, confident in our ability to quickly iterate on products, and deliver consistent value to the business.