Working with microservices: Platforms and patterns

I am a big fan of microservices. And at the same time, I am not always so fond of them. Sometimes working with microservices reminds me of quotes like “small is big” and “slow and steady wins the race.” At the same time, sometimes it reminds me of the quote “too many cooks spoil the broth” too.

Back in 2014, I was working as a Software Engineering Lead for a company that was going through a digital transformation by adopting a microservices architecture. The words digital, transformation, and microservice were all music to my ears. At that time, as an Engineering Lead (I’m currently a Solutions Architect) I was curious to learn about this new pattern. To stay ahead of the game, I read a lot of articles about microservice architecture, spoke to technology leaders from my network, and studied applicable use cases. At this point, I truly believed in microservices architecture and was convinced the monolith was going away. From there on, every company I have worked for has either adopted, or was in the process of adopting, a microservices architecture. However, in a few of those companies the leadership team could not convince the organization why they were going down the microservices path. The general answer was that “Every other company is doing this and people are presenting about microservices as a game changer at every conference, so let’s do this.” Having worked for multiple corporations in various business domains by providing architecture and design support, I see that it takes a great amount of patience, time, and experience to re-architect existing monolithic apps into microservices architecture.

Before we get into my five key takeaways from using microservices, let us take a step back. So what is a microservice? According to one definition, there is actually no standard definition for this architecture style, but there are certain characteristics “around organization, around business capability, automated deployment, intelligence in endpoints, decentralized control of languages and data.” In my opinion, if a company has to exhibit all the above characteristics to use microservices, then we are not just talking about technology transformation, but also a big cultural change within the organization.

Now, below are five takeaways from my experience in working with microservices:

In the traditional software approach, the development team builds a monolithic piece of software and the production support team manages it. In this approach, the new owner (the production support team) often does not have complete knowledge of what went into building the component (such as code logic, technology used, etc...). Their core work is to ensure the production system that meets business needs is up and running. There is usually no defined efficient communication path among the teams. Issues with production systems will either lead to a rollback to a restore point or a fast short-term fix. Sometimes a small production code issue will trigger a whole new process. That adds delays as the problem usually has to be resolved by the original development team.

There is a huge amount of risk if you approach microservices with a waterfall mentality (up front design, centralized process over release schedule, build and deploy). You will probably end up with a much more complex system without enjoying any of the benefits promised by microservices.

In the microservices world, you will often hear the word “product”, not project. In this approach, the stakeholders (Users, Program, Product, Tech) work for a common goal, The Product. Product-mode differs from the project-approach right from funding to software delivery through maintenance. Product has a direct relationship to business capabilities offered. And unlike in the traditional approach where multiple teams are involved in building a single monolithic application, the microservice model allows one team to take full responsibility to build and manage this small unit of software. In product-mode teams are stable, cross-functional, and outcome-oriented; they ideate-build-run. Each team is a single department with a unified reporting hierarchy that builds small chunks of independent software units according to the roadmap. The teams in one tier consider the other tier teams as their customers (internal) and collaborate with them to solve business problems, not work in a scope-delivery manner. Since the engineering team working in product-mode understands the software behaviour in production, any issues will be addressed with immediate resolution, avoiding delays.

Here at Capital One, the You Build You Own (YBYO) concept allows teams to take full responsibility to design, build, test, and deploy software in production. Engineering teams collaborate with products and interact with users directly, with continuous feedback from those users helping the teams to build high quality products.

Takeaway: Smaller scope allows teams to better build and manage microservices. Product-mode enables partnership, ownership and builds strong relationships with end-users.

Before joining Capital One, I worked at another company on a team, building a product catalog service for their company’s eCommerce website. The company was following a microservices approach and this product catalogue service was designed to provide a list of products to the end-user based on a request criteria. As my team had control over data and the catalog database, we chose Java and SpringBoot to build our service (we were comfortable with the programming language and more libraries were available to use). Finally, the service was exposed on an API Gateway for end-users.

Similarly, there were several other teams in the company that built their own services using different technologies. From a product point of view, each feature was supported by various services built on heterogeneous platforms. This model solved a major problem with having to recruit and train teams to build monolithic apps using the same tech stack. In this microservices model, each team could choose tools that were apt for our business needs, and hire new teammates accordingly.

Microservices is an architecture style where components of a business application are built via services. Each service is a logical software unit of business process which is independent of other services. The freedom (no-dependency on other services, choice of technology, etc.) in this approach opens the doors for exploring new technologies, building local software components, and designing based on the scope defined for the service.

At Capital One, software products are aligned with business capabilities. The various lines of businesses (LOBs) build and manage their own products and there is a cross-functional line of business that is focused on building and managing enterprise products such as data lakes and platforms that cater to the needs of all LOBs.

Takeaway: The underlying principle of loose coupling and high cohesion allows teams to build multiple products that solve bigger business problems

A company I worked for before joining Capital One identified a few monolithic business applications that could be migrated to a microservice architecture. There was no change in the product’s vision, as the business functionality of the monolith did not change. More teams were recruited, anticipating that services would need to be owned by these teams. Services were deployed as per the corporate release schedule and the infrastructure team still owned production systems as their process was not impacted by this initiative. Two years after the program started, not much progress had been made and the organization ran out of budget.

This is one of many examples where teams within companies should communicate better over microservices implementation. It's not just app development and new technology that brings digital transformation, a lot of work needs to be done in product analysis, budget estimates, architecture, redesign of deployment procedures, infrastructure scaling, and other relevant processes. Transition to microservices involves time, money, and big changes in how you see business problems.

Takeaway: Microservice is not just an architecture style, it demands a cultural shift that impacts every team in the organization.

The microservices journey will lead to the creation of a lot of products, services, and teams. Organizations have to have a solid roadmap before taking on this complex architecture. That is because strong enterprise products are required to support and bring together those various teams working in a microservice style. For instance, tools that support API documentation, and collaborative tools such as source control management, issue trackers, and tools enable automatic deployments.

Engineering teams build services and expose them as APIs on an API Gateway. These API Gateways are like a marketplace for REST APIs and the backbone of the organization for daily business operations. Once an organization gets on the right track with a microservices approach, a continuous stream of services is created, upgraded, replaced etc., and engineers may not know where every service is located. A Service Discovery system allows services to be detected automatically and help them find each other.

To achieve better performance and fault isolation, components of a microservice need dedicated infrastructure. A microservice should have its own release schedule and should be allowed to deploy to production at any time without depending on others. The choice of an effective tool that supports continuous, real-time monitoring and analysis of the microservices is an essential. APIs are the interfaces to the microservice world so API logging, performance monitoring, and security are also key for the progress of IT services in the organization.

Resilient Microservices can be built following various patterns such as the Retry Pattern, which enables an application to handle transient failures when it tries to connect to a service or network resource by transparently retrying a failed operation. The Circuit Breaker pattern, allows for handling faults gracefully when errors occur connecting to a remote service or resource. This will avoid cascading failure in the microservice ecosystem, which can improve the stability and resiliency of an application. Each service in a microservice is a separate component and every single function or service can be scaled up without having to scale the entire application. The multiple instances of critical services can be deployed for high availability and better performance without impacting the performance of other services.

Takeaway: Though the initial phase of transition to microservices takes a lot of resources and effort, with time and dedication and the help of automation tools, businesses can reap benefits and will be able to deliver quality products to market quickly.

Microservice may not be suitable for every business or use case. This includes ones where a team has to build a simple application with few features or are unable to break large monoliths into smaller pieces or did not understand the trade-offs that come with the microservice architecture style.  Additionally, some businesses may not be provisioned with rapid development and application deployment capabilities or do not need to constantly monitor applications or businesses for which the time for recovery (in case of failure) is high with minimum business impact.

Microservices are only a tool, and like all tools, they are not a solution to every business problem. Business takes precedence over everything, the underlying systems can adapt to any architecture pattern, whether they are monoliths or microservices. Before deciding on microservices, every company has to understand their business needs and review the trade offs before making a decision on microservices.

Capital One invested a lot of time and effort in studying the application of microservices before going all-in with a cloud and microservices architecture. The able leadership, visionary product teams, and the skilled engineering teams collaborated to help Capital One achieve the goal of becoming a technology leader in banking. 

Takeaway: With microservices there is no free lunch.

***

With microservices architecture, there is a huge surge in infrastructure needs, cost and complexity, why are companies adopting microservices? Large companies with large customer bases thrive by delivering quality products to customers within a short amount of time. Their systems need to be always up and running, serving customers distributed across various geographies. Microservices are an approach that helps achieve this goal. In today's world, with the advent of cloud-native infrastructure, automation of the delivery pipeline (DevOps), and adoption of containers, companies should look into the advantages of microservices.

I would like to affirm that you should not adopt something just because others are doing some cool stuff with it. Instead, invest the time and effort to understand this architecture pattern and its relevance to your company before adopting microservices. I hope my key takeaways have left you feeling more prepared for your microservices journey.