A new mindset for the cloud: Migrating from on-premise

The year was 2016 and I was in the middle of an intense interview session for a software engineer role at Capital One. The interviewing director asked me,

“So, tell me what you know about AWS.”

I truthfully replied, “I can spell it and that’s about it.” Thankfully, he laughed and I then proceeded to answer his question by telling him my theoretical understanding of distributed computing and how Amazon might have leveraged it in building their platform. Needless to say, I got the job. What transpired next would alter my career path and force me to lift my engineering mindset from on-premise to the cloud.

When I started at Capital One, I had zero cloud experience, let alone AWS experience. When you start, the number of services offered by AWS is daunting. AWS offers over 200 fully featured services which range from database services to virtual servers to storage services. The first phase of migrating my mindset was to get a clear mapping of the components I knew from the on-premise world and what their equivalent components in this new world of the cloud were. After a few web searches and going through AWS' FAQs I was able to make a quick mapping of some basic components I used on-premise and their equivalent in the cloud.

Server →  EC2 (Elastic Compute Cloud)

SQL Database →  RDS (Relational Database Service)

Data Center →  VPC (Virtual Private Cloud)

File Server →  S3 (Simple Storage Service)

Tape Backup → Glacier

This approach is commonly referred to as “lift and shift” when migrating applications to the cloud. During my pre-cloud days I used to interact with MySQL and MS SQL Server on premise. This involved working with the IT department to procure hardware and database licenses. The total time to implement the database system could take anywhere from 1 to 3 months. However with a service like Amazon RDS, I was able to get a database up and running in minutes. Now that I was able to see how AWS services mapped to the existing components I used on-premise, the next phase in my mindset migration was to see what cloud native components existed that I was not able to get on-premise.

The power of AWS is in the native features that were built to operate solely in the cloud. Although AWS has a plethora of native cloud services, there were three services I had to get familiar with very quickly in order to be able to leverage my cloud adoption journey to its fullest potential. These were:

• AWS CloudFormation
• AWS Lambda
• AWS CloudWatch

Let’s get into each of them.

Infrastructure as code is one of the key tenets in building applications in AWS. AWS CloudFormation provides you a way to manage your whole infrastructure as code in JSON or YAML format. Using infrastructure as code enables you to version control your environment and also easily build up and tear down environments for testing, QA, prod, etc.

Running code without having to think about servers is a very powerful feature provided by AWS Lambda. All you have to do is upload your code and AWS Lambda will take care of executing the code when you want it triggered. It can then be triggered by other AWS services or directly by a REST API call. AWS Lambda also supports a large variety of languages from Java and C# to Go and Python. This makes it a very flexible yet powerful feature which easily attracts developers from a wide range of different programming language backgrounds.

AWS CloudWatch is a service that enables you to observe a variety of metrics associated with your AWS cloud resources. It can be configured so you get a notification if certain thresholds are reached with any of your cloud resources.

After understanding how these cloud native features worked, it was time to see how to glue them all together.

AWS is continually improving existing services and releasing new ones. It can be a challenge trying to keep up with all these changes in the ecosystem. Here are some of the ways one can keep up:

• Certifications: AWS currently offers twelve certifications that range from entry level to advanced level, as well as specialized certifications for things like machine learning and data analytics. These certifications require you to recertify every couple of years in order to maintain your certified status. The goal of getting certified should not be to accumulate as many certifications as possible, but to be able to prove that you understand the underlying workings of AWS as it applies to your role. Focus on the certification, or certifications, that deal with your area of work first. I got certified at the Associate and Professional level for the Cloud Architect, Developer, and SysOps as these certifications were most in line with my role at Capital One.
• Conferences: AWS hosts a variety of conferences every year from the popular AWS re:Invent to the AWS Summit. These conferences are a great place to meet other professionals working in the same field and engage in hands-on workshops to keep your skills fresh and up to date. I was fortunate to attend AWS re:Invent in 2018 in Las Vegas. With over 50,000 people in attendance, it was a very unique experience networking and learning all the ins and outs of AWS services. It was at the conference I got fully engaged with Machine Learning and AWS SageMaker. I was able to build a proof of concept system and wrote a blog about my experience with AWS SageMaker.
• White Papers: Every AWS service has at least one white paper associated with it. Some of these white papers can be long, but they contain a wealth of information,  especially if you are trying to use a service you are not that familiar with.
• Practice! Practice!! Practice!!!: None of the above will stick without practice and getting your hands dirty with AWS. If you are unable to work with AWS at your job, you can sign up for a free tier account that gives you access to most of the popular services to whet your appetite on personal projects.

Finally I can see where the Cloud was leading me, but how do I grow in this new Technology landscape?

The last phase explores how to “think in the cloud”.

You won't travel far in the cloud landscape before you come across the term DevOps. This is a set of practices that has helped convert some of the traditional on-premise software development lifecycle, like the Waterfall Model, into a tighter loop of constant feedback. AWS has a variety of tools to help facilitate the DevOps mindset. Some of the key facets where thinking in the cloud differs from on-premise are as follows:

• Infrastructure as Code, (IaC): There is an analogy in cloud circles that on premise servers are like pets. It’s true, you can have an affinity for them because they are usually a big clunk of metal you can see, and even hear its humming, in your datacenter. In other words, it’s something physical and present, not something you can easily dispose of. In the cloud however, servers become like a herd of cattle. You can easily grow your herd of servers (scale out) as demand rises and you can thin out your herd as demand drops (scale in). Servers in the cloud are no longer physical constructs, but Infrastructure as Code with AWS tools like CloudFormation templates helping facilitate this.
• Microservices: Before the advent of the cloud it was already common practice to decompose monolithic applications into smaller verticals called microservices. However, AWS has made it easier to implement microservice architectures with its wide array of serverless services. Now the developer does not have to worry about managing servers, but can just simply deploy the code to solve the business need.
• Continuous Integration and Continuous Delivery (CI/CD): This is one of the key tenets when it comes to DevOps. This is the ability to constantly push code through a pipeline all the way to production. The code goes through stages where automatic actions like code quality checks, unit testing, integration testing and security checks are performed. Manual approval gates can be added to the pipeline to have a person “click a button” to allow any code changes that have met all the requirements to proceed to production. AWS offers an array of developer tools to seamlessly implement CI/CD.
• Monitoring and Logging: After writing and deploying your application, you need to have constant monitoring of the application so that your team can be alerted of any state you feel will impact the business goal. Using the AWS tools under the CloudWatch family, you can achieve a high level of monitoring and insight into how your application is running.

AWS has provided a flexible and scalable platform that has made it easier for companies and individuals to successfully migrate to, and build in, the cloud. My journey into the cloud is not about a destination, because there is never going to be one landing spot. AWS is constantly improving existing services and introducing new ones to solve the new problems we are trying to solve in the cloud.  It’s been over four years since I started my own journey with AWS and I still find it an exciting and challenging space to work in. I hope this inspires you to take the plunge as well.

Background photo created by whatwolf - www.freepik.com

The ease of creating resources in the cloud can lead to some unintended consequences. Just like with every tool, knowing how to use it properly will determine how successful you are in accomplishing your goals.

I have found that the AWS Well-Architected framework is a good guideline to follow when building applications and infrastructure in the cloud. The framework lays out five pillars:

• Operational Excellence: This involves running, monitoring and fine tuning systems to deliver business value. Cloud services like AWS Trusted Advisor and AWS Support are key services for this pillar.
• Security: This focuses on protecting cloud resources and data. AWS offers various security solutions which are essential for this pillar.
• Reliability: This is the ability for a system to prevent and quickly recover from failures while still meeting business goals. AWS CloudWatch and AWS Trusted Advisor are key services for this pillar.
• Performance Efficiency: This pillar focuses on utilizing the right cloud resource and configuration for the right job. AWS Auto Scaling and AWS CloudFormation are key services for this pillar.
• Cost Optimization: This pillar focuses on monitoring and fine tuning cloud resources to produce maximum business value at the lowest cost.  AWS CloudWatch is a key service for this pillar.

The AWS Well-Architected framework can be a lot to read and daunting to get started with, but familiarizing yourself with it, even at a high level, will serve you well as you start to build applications in the cloud. I have learned through my cloud journey to lay emphasis on Operational Excellence and Security when building any cloud architecture. The other three pillars come naturally when these two are done right from the start.

Now that I understood the basics of building in the cloud, the next phase in my mindset migration was to keep up with the ever-changing landscape of AWS.