Once you build a website, you need to make it available to its users . That process is deployment. Historically, it's been basically putting your application on a server. Nowadays, I'm sure you've heard of containers, orchestration, and clusters. In this introduction, we will see what problems those tools and concepts solve and why application infrastructures have got more complex than a simple server.
Let's start from a simple setup. You deploy your website on a single server that you manage. The server is powerful enough to handle your customers requests and everything is fine. Now imagine you need to update the runtime of the server, for any legit reason. You have to shut down the server and your app is now down during your operation.
It would be better to have another server that can assume the workload while you are updating your main server. Good, here is a second server. Your clients are now still served and you just finished the maintenance of the first server which is back running, so you shut down the second server.
You now see the value of having a backup server in such cases. So you decide to keep the second server around for next time. Actually, anytime the first server would be unable to serve clients. For any reason.
Now you have a backup server. Your main server outage can be predictable, like a maintenance as we saw, but it can be unpredictable, in which case you might not be around to switch the workload manually.
It would be nice to have a system that tracks the capacity of the main server to serve requests, and automatically transfer them to the backup server if the performance of the first server drops. A failover mechanism.
Good, now you have a much more resilient setup. However, you've noticed that most of the time only one server is working, but you have two of them. Wouldn't it be better if both of them can work together? Like each one handles half of the requests? You would be able to serve twice as many clients.
Perfect! You've set up a load balancer in front of the two servers to do that.
After a moment, you feel like this could be easily scaled. After all, the more servers you have, the less impact one server outage has on your application performances. So you decide to add three servers to your cluster and register them with your load balancer.
The five servers allow you to serve 5 times more clients than initially, and you only lose 20% of the total capacity when you do your regular runtime updates, compared to 100% initially.
You are happy.
Couple months later, you have developed a new version of your application that you want to deploy in place of the previous version. This should not be a problem, you just install the new version on your instances (servers) one by one. You even wait a little after every installation to make sure that nothing breaks before you move on. If anything broke, you would see it on the instance, and you would not continue. This cluster is definitely useful, not only to scale the capacity and resist failure, but it also allows you to test new versions under real workload, thus allowing safe deployments.
You have made a good choice with that infrastructure.
Couples months later, you developed a brand new app that you want to deploy on the same model. But you don't have room for 5 other servers. It's pretty expensive, and your existing five servers are far from their maximum compute capacity for the current app. You are thinking it would be very nice to be able to deploy your new application on the same cluster. After all, an app is just a set of programs, why couldn't you just add another set of programs to the same instances?
Well you can, and that's actually pretty usual. You need to make sure that the different sets of programs will not interfere with each other. For example, you don't want your second app to use all the resources available on the instance. That could harm your first app. Similarly, you don't want your first app to accidentally override a file the second app needs. Basically, both of your apps need to be contained.
A very straightforward way to contain apps is to put them into... containers. This also gives the ability to define the whole OS environment for the app. Why does this matter? You could deploy a Linux app alongside a Windows app on the same instance running on whatever OS supports the container technology you are using.
Containers allow you to use your cluster compute to the fullest, without worrying about conflicts between your apps.
Now we are a year later and your two apps are now very successful, and they keep growing. You eventually added 5 more instances to your cluster. But here’s the thing. You notice that even if your infrastructure scales well, something feels wrong with your apps: Every time you change a part of one of them, you have to deploy the entire codebase. And every time a small part of the app causes a runtime error, the whole app crashes. Also, lately you notice that the apps requirement in compute is increasing. That's why the 5 new servers you bought were much more powerful. But you feel like only a single part of the app needs a lot of compute. The more it goes, the more it feels like several parts of your apps have different needs, different lifecycle, even different ownership. The only reason they are part of a single app is that they have to communicate together.
You decide to split your apps into several parts, deployed and running on their own runtime but that communicate with each other through a well defined API. That's a service oriented architecture.
Now you can deploy each service independently. Resource hungry services can be placed on more powerful instances. Once again, you have optimized your resources, and solved the issues you had with your apps when they were monoliths. However, now the infrastructure is much more complex. It's harder to keep track of what service runs on what instance. Also, now your instances are not simple replications of one another anymore. You decide to replicate some services depending on their importance.
Now some services are replicated 3 times, others are replicated 4 times, others are not replicated at all. It’s seems like chaos.
You can still reason about what should run in your system, but actually managing it becomes a different story. You don’t think in terms of single machines anymore, but you also can’t realistically manage everything by hand on each instance.
If a service needs 3 replicas, you now have to manually make sure 3 instances are running somewhere in the cluster. If one disappears, you need to notice it and recreate it. If you add a new instance to the cluster, you need to decide what should run on it. Every change becomes a coordination problem across multiple machines and multiple services at the same time.
Even deployments become harder. Updating a service is no longer about replacing a single running program. It’s about carefully rolling out changes across multiple replicas, across multiple instances, while making sure you don’t break communication with other services that depend on it. And you have to do this repeatedly, for every service in the system.
At this point, the real difficulty is no longer running code on servers. It’s keeping the entire system consistent over time. Making sure the right number of service replicas are running. Making sure they are distributed across your instances in a sensible way. Making sure new versions are introduced cleanly, and broken ones are removed.
So instead of continuing to manage all of this manually, you start thinking in terms of describing what you actually want.
You don’t want to say: “start this service on that server.”
You want to say: “this service should always have 3 instances running, this other one should have 4, and they should all stay distributed across the cluster.”
And then something else takes care of making reality match that description.
This is where orchestration comes in.
Orchestration is the layer that sits on top of your cluster and takes care of running all these services according to the state you define. Instead of manually placing services on instances, scaling them, or restarting them when something changes, you declare the desired state of your system, and the orchestrator continuously works to keep it that way.
It turns your cluster from something you operate manually into something that operates itself based on your instructions.
In this introduction, you've had a glance at infrastructure maintainance and deployment constrains. Hopefuly, you now understand the real world needs behind containerization, microservices, and orchestration. The following chapter will deep dive into every aspect we've covered and their implementation in AWS. We know your time is valuable so we'll try to provide only relevant information, in the simplest way. The lessons are ordered lists of simple and straight-forward facts. It's also important to note that this chapter covers all the AWS Developer DVA C02 requirements on deployment and beyond.
Table of content: