Scionova

My experience from Cloud Development

Roughly one year ago I began developing in AWS (Amazon Web Services). I could then immediately see what all the buzz was about. AWS, and other Cloud platform providers (Azure, GCP etc.), supplies an immense amount of services. These services include both managed and unmanaged, from completely serverless to a provided dedicated server. One where the customer has (almost) complete control. These services provide the tools to develop a scalable, secure, agile and cost-efficient application where hardware, servers and OS management can be abstracted away to allow more focus on the actual code of your application.

In the past year I have been involved in the development of a microservice in AWS, and the past month have been spent on preparation for, and later passing, the AWS Certified Solutions Architect – Associate exam.

 

What problems could be solved by using a Cloud Platform?

Imagine that you are part of a start-up aiming to develop a platform that provides a marketplace for autonomous car owners to make their vehicle available for hire when they are not using it themselves. Traditionally, starting the development of the application requires servers to run the application and databases, as well as other hardware, e.g. network equipment to enable connection to the internet.

As development continues, changes in requirements might need additional hardware to be purchased or existing hardware to be replaced. Building tools and other CI-related tools are required to ensure the quality of your platform, all which need their own servers. Moving the solution to production might require another set of equipment with capacity to handle spikes in the load of a production environment.

To minimize downtime of the platform, back-up servers will have to be purchased. All this equipment is associated with a capital investment which might not be available for your start-up without the assistance of an external investor. In the cloud, services are paid for in a pay-as-you-go model. No up-front investment is required, which better fits the financial situation of your start-up.

 

Serverless Architecture

For the last couple of years, ‘Serverless’ has had a huge impact on the industry. But what does it really mean to have a Serverless Architecture? Traditionally, applications have been installed on a specific physical server, where everything from hardware to application requires maintenance. With a serverless architecture, the only concern for the developer is the code of the application. The rest is left to the cloud provider to maintain. Of course, this does not mean that code is not running on a server, only that it is not important which server the code runs on.

Other than reduced maintenance, a serverless architecture brings automated scaling of your environment, a cost model that depends on the actual utilization of the services and a more microservice-friendly infrastructure.

 

AWS Services

These are some AWS services which could provide an entry point for someone who wants to get acquainted with the AWS ecosystem:

 

•       AWS CloudFormation (https://aws.amazon.com/cloudformation) – An effective tool to describe your AWS infrastructure resources as code (YAML, JSON) in CloudFormation templates. This enables you to store the specification of your entire infrastructure in version control. When changes are made to the template, CloudFormation will calculate the delta from the deployed set of resources into a change set and thereafter execute the change set. Having your infrastructure as code is also crucial when managing multiple staging environments (Dev, QA, Prod etc.) where it is important that the environments are identical to facilitate code and infrastructure quality assurance. An alternative to CloudFormation is Terraform, which is an open-source, platform independent tool to describe infrastructure (https://www.terraform.io).

 

•       AWS Lambda (https://aws.amazon.com/lambda) – This is the core of any serverless application developed in AWS. A Lambda basically consists of the code that you want to execute and something that triggers it, e.g. an API is called, or a database table is updated. When the Lambda is triggered the supplied code is deployed and executed and shortly after the execution is finished the deployment will be removed. Any parallel triggering of a Lambda will result in multiple deployments of the code and this will scale infinitely. The cost of a Lambda is based on the number of executions and a combination of the execution time and the amount of memory that is allocated for the Lambda code.

 

•       Amazon EC2 (https://aws.amazon.com/ec2) – Elastic Compute Cloud – EC2 is a service that provides a virtual machine, or “instance”, on a server in AWS. When deploying an instance, it is possible to choose from an abundance of instance types and pre-configured operating systems with different application-setups. An instance type can be anything from cheaper general-purpose instances to more expensive instances, e.g. an instance equipped with more graphics resources to enable more graphics-intensive workloads or machine learning workloads. Each instance type also has several different sizes to support workloads of varying load.

 

•       Amazon VPC (https://aws.amazon.com/vpc) – Virtual Private Cloud – VPC is used to set up a network in AWS. The network can then be equipped with subnets with different CIDRs, NAT gateways, Internet Gateways, Load Balancers, services to connect the network to an on-premise network etc. All without setting up any hardware yourself. EC2 instances can be deployed in public and private subnets to provide a tiered application setup where database instances and back-end instances (in the private subnet) are only accessible through the front-end instances (in the public subnet).

 

•       Amazon ECS (https://aws.amazon.com/ecs) – Elastic Container Service – A container orchestration service provided by AWS that supports docker containers. The service is available in two modes, EC2 and Fargate. The EC2 mode requires the developer to manage the EC2 instances that the containers run on as well as the scaling of the number of the instances, while the Fargate mode is fully managed in this regard. An open-source alternative to Amazon ECS is Kubernetes (https://kubernetes.io).

 

•       Amazon DynamoDB (https://aws.amazon.com/dynamodb) – DynamoDB is a fully managed NoSQL database that provides great performance and is highly scalable. DynamoDB together with AWS Lambda and Amazon API Gateway (https://aws.amazon.com/api-gateway) provides all the tools required to build a small, simple and completely serverless microservice. A NoSQL database is an excellent fit for applications with well-defined database access patterns, i.e. the queries that will be executed are known at the design phase and the NoSQL table(s) can be designed thereafter. However, for applications with more ad hoc database access patterns a SQL database would be a better choice. This session from AWS re:Invent 2018 (https://www.youtube.com/watch?v=HaEPXoXVf2k) makes a deep dive into DynamoDB and explains when a NoSQL database should be utilized.

 

•       Amazon S3 (https://aws.amazon.com/s3) – Simple Storage Service – A managed object storage service which provides a whopping 99.999999999% of durability for any stored object, which is achieved by storing the objects in multiple data centers across a Region (a Region comprises a set of Availability Zones, which in turn comprises a set of data centers). S3 is used for storing different types of files, e.g. videos, images and documents. Some features of S3 includes versioning of objects, replication of objects to another AWS Region, archiving objects to cheaper storage options when the object is no longer frequently accessed (e.g. Amazon S3 Glacier https://aws.amazon.com/glacier), hosting static web content etc.

 

Some things to be aware of

As mentioned, cloud development offers lots of opportunities moving forward. However, there are some things to take into consideration when deciding on whether to move your solution to the cloud. Platform lock-in is one thing. Deciding on a cloud provider will most likely make you dependent on that company’s solution, which will make you vulnerable to any changes in price or functionality of the used services.

Another consideration is the difference in price model from traditional development. Services in the cloud, especially managed services, are often priced per invocation and/or per unit of data transferred/processed/stored. This model might not suit solutions that have an even, non-fluctuating load over time, in that case a fleet of EC2-instances could be a better fit. Neglecting this detail when designing a solution could result in unnecessarily high costs for the solution.

Being well-prepared when designing a solution is key to avoid these pitfalls. The AWS Well-Architected Framework (https://aws.amazon.com/architecture/well-architected) provides some guidelines on how to architect a solution with performance, security, cost etc. in mind.

 

If you are interested in starting your journey in the cloud in general and AWS in particular and are eager to learn more about the possibilities and risks of cloud development, do not hesitate to contact me at Daniel.Andersson@scionova.com.

 

If it’s not already, this blogpost will give you some practical tips on how to make your debugger your best friend. If you are new on the job and thrown into a big legacy system, it can sometimes be really difficult to understand the flow of the code. Even when you are familiar with the codebase a particularly complicated piece of code may leave you stumped. A debugger can be a helpful tool to understand what is happening and where information stems from. Going back to the basics and really getting to know how the debugger(s) in your IDE(s) work can be a real boost. 

 

This blog post is not supposed to be a tutorial for any one IDE. Instead I will go through debugging concepts that are present in most IDE, as well as some nifty tricks found in specific debugging tools. I am most familiar with the Eclipse debugging tool, so most of my references will point to there. However, the concepts that I cover are present in most debugging tools. 

 

Get to know your break points

Debugging is stepping through your code, line for line, while being able to monitor the changes in variables in your code. Most of the time you are not interested in each and every line of code. You are interested in a specific part of the code or a specific variable. As such, you want to be able to tell your debugger when it should pause execution for closer inspection. You do this with breakpoints. 

 

The most basic breakpoint will simply stop the execution at the given line; however, you can do much more with breakpoints! One of my mostly used breakpoints is the conditional breakpoint. As the name suggests it will halt execution when a given condition is met or when a value changes. This allows you to focus on what is really important. 

 

A really helpful break point is the exception breakpoint. If you have no idea why an exception is thrown and you want to find out the cause, exception breakpoints will halt execution right when the exception is triggered. It is then easy to see what triggers the exception, and what the cause of that trigger is. 

 

If you are working with a multi-threaded application and you want to follow the execution of a specific thread, you may want to filter a breakpoint on a specific thread. Then you can follow the execution of that thread without having to worry about being confused by another thread. 

 

You can also do some additional things with breakpoints, such as suspending execution when the breakpoint has been hit a certain number of times or suspend all breakpoints until a certain breakpoint has been reached.

 

Get to know the controls 

When you have reached the code of interest for debugging, you may want to follow the execution more closely. To do this you have some progression controls at your disposal. 

 

The basic progression controls for debugging are fairly self-explanatory. The step into option will go into the statement you are currently at. The step over option in contrast, will jump over the statement and show you the result after. This means that if you choose to step over a big method, you do not have to follow it through the entire method, you will only see the result. 

 

The final basic progression tool is the step return, which will take you out to the caller of the current statement. If you are in a method this would mean that you would go to the place where the method was called. 

 

A lesser known, but very useful progression option is the drop to frame option. Using this will take you back to the beginning of the current frame you are in. In the case of a method, this would mean you will be taken back to the very top of the method. This can be very useful while finding out where an issue originates from. You might step over a piece of code with the step over function and find that a method call is the root of your error. Unfortunately, step back is generally not an option while debugging. Drop to frame will take you back to the beginning of the code piece you are currently in though, and you can choose to go into the statement that caused the error again.

 

The drop to frame option is also very useful if your IDE and debugger allows for hot changes. That is, they allow you to change the code while debugging without having to recompile and rerun your application. If you are working on a big project that takes a long time to compile or run this can be a great time-saver. In Eclipse while working with Java, you can do code changes that do not affect method headers and similar big changes. After doing a change and saving you will automatically be brought to the top of frame, allowing very quick feedback for your change.

 

If you are working with applications that might be triggered by another application, you may think that you cannot debug your application properly. These kinds of situations can be really difficult, as it is not the application that you want to debug that drives the execution. However, if you are developing in Visual Studio you are in luck! Visual Studio allows you to attach a process to your debug session. This will in turn trigger the execution in the application you are debugging. To do this go to Debug > Attach to process and select the process to which you want the debugger to attach. 

 

Make friends 

While debugging may seem basic, and the prospect of debugging your own or others code may be daunting, I have found that it really helps me understand the applications that I am developing. While I was still a student, I rarely saw the use of the debugger, it was mostly just a bothersome tool that never wanted to work for me.

 

When I started working in a bigger project with more complicated data structures I realized how useful it can be. Getting familiar with the tools and options available to me showed me how essential good debugging skills are to my work as a developer. It takes some getting used to, but as the saying goes, practice makes perfect, and these days the debugger is my most beloved development tool. I hope this blogpost has given you some food for thought and might have introduced some concepts that you were not aware of before. 

 

Best regards ,

Lisa