Embracing AWS Lambda and serverless architecture

Capital One has been on a powerful tech transformation journey for many years.In 2020, we closed our last data center and moved all-in to the Cloud, with AWS as our primary cloud provider. The Cloud has allowed us to move faster and focus on building the applications that provide value to our customers. Today, we are focused on modernizing our tech stack by adopting serverless-first architectures.

We no longer need to manage physical data centers and infrastructure, but our scale requires us to manage AWS accounts and resources in those accounts. At the beginning of our cloud journey, we deployed our applications to Amazon EC2. This means our developers were responsible for managing the entire tech stack. Developers had to deploy EC2 instances, manage the scaling, configure the autoscaling policies and manage the Operating Systems and Containers deployed to these EC2 instances. We had to deal with patches, vulnerabilities and other tasks we call “Run the Engine” (RTE). We quickly realized RTE costs are significant for all teams.

This is one of our driving factors in why we decided to adopt serverless approaches. Today, our preferred choice of computing is AWS Lambda functions or Containers with ECS Fargate. Lambda allows our developers to focus on building the application logic that delivers value to our customers. We let AWS handle all of the other tasks that cost us time but do not add value.

We operate within AWS at a massive scale – we have thousands of accounts and tens of thousands of Lambda functions. We learned it’s really important to set both development standards and account management standards. However, Capital One is a large enterprise made of multiple Lines of Businesses (LOBs) and decisions that affect the entire enterprise should not be made in a silo. This led to the launch of our Serverless Center of Excellence (CoE). Our COE is made up of representatives from each LOB – we meet on a regular basis to talk about serverless best practices and set standards that work for the entire enterprise.

Some things we learned from our journey include:

• You probably need new tools. AWS SAM is an excellent developer tool that allows you to build and iterate locally.

• Lambda shifts the unit of scale from requests/second to concurrency and this will be a new concept to all. Make sure your developers understand how to calculate concurrency.

• Multi account/small accounts are better than a few large accounts. This strategy allows you to limit blast radius and provide each team with their own accounts.

• Make sure you set development standards. For example, we require a LIVE_TRAFFIC alias to be the entry point for all Lambda functions. This ensures that we understand how all Lambda functions are executed. We also require all functions deploy using versions – this ensures consistency and prevents accidental changes.

Finally, here are the top AWS Lambda architecture best practices that have helped us build better serverless applications in the cloud:

1. Optimize your memory. Use Lambda Power Tuner. Memory = $$$

2. Set cloudWatch Log Retention. Logs are not free!

3. Use Lambda Powertools. Make common tasks easy.

4. Optimize for the cold/warm lifecycle.

5. Use provisioned concurrency with autoscaling.

6. Understand and monitor metrics.

7. Use AWS SAM. Build, emulate, test and debug all locally.

8. Use the right AWS SDK!

   • Java: v2

     • Use the right HTTP Client (Netty, Apache, AWS CRT**)

   • Node: v3

   • Go: v2

9. Use Snapart. It’s free.

10. Set standards. Tags, aliases, IAM permissions, etc.

Capital One's journey to AWS Lambda serverless architecture marks a significant step in our continuous pursuit of innovation and efficiency in the tech landscape. The transition from data centers to the AWS Cloud and adoption of a serverless-first approach exemplify the power of embracing cutting-edge technologies. 

The establishment of the Serverless Center of Excellence and adherence to AWS Lambda architecture best practices further demonstrate our commitment to excellence and collaboration within a massive-scale environment. These shared experiences serve as an insightful guide for those looking to leverage the potential of serverless computing and enhance their cloud-based operations.

New to tech at Capital One? We’re all in on the cloud, serverless and open source:

• See how we’re building and running serverless applications at a massive scale.

• Explore open tech jobs and join our world-class team in changing banking for good.