My experience from Cloud Development
Roughly one year ago I began developing in AWS (Amazon Web Services). I could then immediately see what all the buzz was about. AWS, and other Cloud platform providers (Azure, GCP etc.), supplies an immense amount of services. These services include both managed and unmanaged, from completely serverless to a provided dedicated server. One where the customer has (almost) complete control. These services provide the tools to develop a scalable, secure, agile and cost-efficient application where hardware, servers and OS management can be abstracted away to allow more focus on the actual code of your application.
In the past year I have been involved in the development of a microservice in AWS, and the past month have been spent on preparation for, and later passing, the AWS Certified Solutions Architect – Associate exam.
What problems could be solved by using a Cloud Platform?
Imagine that you are part of a start-up aiming to develop a platform that provides a marketplace for autonomous car owners to make their vehicle available for hire when they are not using it themselves. Traditionally, starting the development of the application requires servers to run the application and databases, as well as other hardware, e.g. network equipment to enable connection to the internet.
As development continues, changes in requirements might need additional hardware to be purchased or existing hardware to be replaced. Building tools and other CI-related tools are required to ensure the quality of your platform, all which need their own servers. Moving the solution to production might require another set of equipment with capacity to handle spikes in the load of a production environment.
To minimize downtime of the platform, back-up servers will have to be purchased. All this equipment is associated with a capital investment which might not be available for your start-up without the assistance of an external investor. In the cloud, services are paid for in a pay-as-you-go model. No up-front investment is required, which better fits the financial situation of your start-up.
Serverless Architecture
For the last couple of years, ‘Serverless’ has had a huge impact on the industry. But what does it really mean to have a Serverless Architecture? Traditionally, applications have been installed on a specific physical server, where everything from hardware to application requires maintenance. With a serverless architecture, the only concern for the developer is the code of the application. The rest is left to the cloud provider to maintain. Of course, this does not mean that code is not running on a server, only that it is not important which server the code runs on.
Other than reduced maintenance, a serverless architecture brings automated scaling of your environment, a cost model that depends on the actual utilization of the services and a more microservice-friendly infrastructure.
AWS Services
These are some AWS services which could provide an entry point for someone who wants to get acquainted with the AWS ecosystem:
- AWS CloudFormation (https://aws.amazon.com/cloudformation) – An effective tool to describe your AWS infrastructure resources as code (YAML, JSON) in CloudFormation templates. This enables you to store the specification of your entire infrastructure in version control. When changes are made to the template, CloudFormation will calculate the delta from the deployed set of resources into a change set and thereafter execute the change set. Having your infrastructure as code is also crucial when managing multiple staging environments (Dev, QA, Prod etc.) where it is important that the environments are identical to facilitate code and infrastructure quality assurance. An alternative to CloudFormation is Terraform, which is an open-source, platform independent tool to describe infrastructure (https://www.terraform.io).
- AWS Lambda (https://aws.amazon.com/lambda) – This is the core of any serverless application developed in AWS. A Lambda basically consists of the code that you want to execute and something that triggers it, e.g. an API is called, or a database table is updated. When the Lambda is triggered the supplied code is deployed and executed and shortly after the execution is finished the deployment will be removed. Any parallel triggering of a Lambda will result in multiple deployments of the code and this will scale infinitely. The cost of a Lambda is based on the number of executions and a combination of the execution time and the amount of memory that is allocated for the Lambda code.
- Amazon EC2 (https://aws.amazon.com/ec2) – Elastic Compute Cloud – EC2 is a service that provides a virtual machine, or “instance”, on a server in AWS. When deploying an instance, it is possible to choose from an abundance of instance types and pre-configured operating systems with different application-setups. An instance type can be anything from cheaper general-purpose instances to more expensive instances, e.g. an instance equipped with more graphics resources to enable more graphics-intensive workloads or machine learning workloads. Each instance type also has several different sizes to support workloads of varying load.
- Amazon VPC (https://aws.amazon.com/vpc) – Virtual Private Cloud – VPC is used to set up a network in AWS. The network can then be equipped with subnets with different CIDRs, NAT gateways, Internet Gateways, Load Balancers, services to connect the network to an on-premise network etc. All without setting up any hardware yourself. EC2 instances can be deployed in public and private subnets to provide a tiered application setup where database instances and back-end instances (in the private subnet) are only accessible through the front-end instances (in the public subnet).
- Amazon ECS (https://aws.amazon.com/ecs) – Elastic Container Service – A container orchestration service provided by AWS that supports docker containers. The service is available in two modes, EC2 and Fargate. The EC2 mode requires the developer to manage the EC2 instances that the containers run on as well as the scaling of the number of the instances, while the Fargate mode is fully managed in this regard. An open-source alternative to Amazon ECS is Kubernetes (https://kubernetes.io).
- Amazon DynamoDB (https://aws.amazon.com/dynamodb) – DynamoDB is a fully managed NoSQL database that provides great performance and is highly scalable. DynamoDB together with AWS Lambda and Amazon API Gateway (https://aws.amazon.com/api-gateway) provides all the tools required to build a small, simple and completely serverless microservice. A NoSQL database is an excellent fit for applications with well-defined database access patterns, i.e. the queries that will be executed are known at the design phase and the NoSQL table(s) can be designed thereafter. However, for applications with more ad hoc database access patterns a SQL database would be a better choice. This session from AWS re:Invent 2018 (https://www.youtube.com/watch?v=HaEPXoXVf2k) makes a deep dive into DynamoDB and explains when a NoSQL database should be utilized.
- Amazon S3 (https://aws.amazon.com/s3) – Simple Storage Service – A managed object storage service which provides a whopping 99.999999999% of durability for any stored object, which is achieved by storing the objects in multiple data centers across a Region (a Region comprises a set of Availability Zones, which in turn comprises a set of data centers). S3 is used for storing different types of files, e.g. videos, images and documents. Some features of S3 includes versioning of objects, replication of objects to another AWS Region, archiving objects to cheaper storage options when the object is no longer frequently accessed (e.g. Amazon S3 Glacier https://aws.amazon.com/glacier), hosting static web content etc.
Some things to be aware of
As mentioned, cloud development offers lots of opportunities moving forward. However, there are some things to take into consideration when deciding on whether to move your solution to the cloud. Platform lock-in is one thing. Deciding on a cloud provider will most likely make you dependent on that company’s solution, which will make you vulnerable to any changes in price or functionality of the used services.
Another consideration is the difference in price model from traditional development. Services in the cloud, especially managed services, are often priced per invocation and/or per unit of data transferred/processed/stored. This model might not suit solutions that have an even, non-fluctuating load over time, in that case a fleet of EC2-instances could be a better fit. Neglecting this detail when designing a solution could result in unnecessarily high costs for the solution.
Being well-prepared when designing a solution is key to avoid these pitfalls. The AWS Well-Architected Framework (https://aws.amazon.com/architecture/well-architected) provides some guidelines on how to architect a solution with performance, security, cost etc. in mind.
If you are interested in starting your journey in the cloud in general and AWS in particular and are eager to learn more about the possibilities and risks of cloud development, do not hesitate to contact me at Daniel.Andersson@scionova.com.
