Part of my job involves speaking to fellow engineers at meetups and conferences about new technologies, evangelising the latest shiny tools and processes for fun and profit.
(Updated December 2019: It’s not technically part of my job any more, now I just put myself through this for GDG Bristol)
These are usually things that fall under the loose, broad and rather fuzzy umbrella of “DevOps”; things like: Cloud, Containers, Serverless, CI/CD, Functions etc. And I’ve noticed that as the DevOps movement itself continues to gain momentum at a runaway pace, the list of things like this keeps growing, and a perceived barrier of entry grows with it.
I know great seasoned engineers who are put off this new-fangled stuff because of this barrier. People think there is too much to learn. But don’t panic! I’m here to let you in on some closely-guarded industry secrets:
A lot of this stuff isn’t actually that new. The shiny tools and processes are just different abstractions of stuff that has been done before.
Like any other thing you do as an engineer, you can get to grips with most of these things within a day or so of self study.
This point is here to emphasise the previous point. I know you don’t believe me right now, but you couldn’t dynamically load kernel modules the first day you used Linux could you? But it didn’t stop you learning how!
I think many dyed-in-the-wool sysadmins may be surprised to learn that all the polishing, automating and yes, actual software engineering they’ve been doing as part of their day job for years just has a new fancy name now (actually it has several, but that’s a subject for another post). So to help those engineers jump over the mental barrier into this brave new world, here’s a quick cheat sheet. Let me know if I’ve missed anything important!
Containers
VMs but smaller (I’m going to get a lot of hate for that!). Actually, just an operating system, application and its supporting libraries packaged into an image that runs in an isolated process on a system. Linux has done this for years with cgroups, before that Solaris did it with zones (now I’m showing my age).
Docker
Technically “Docker” is the hip company that created a toolset and ecosystem to make it easier to run Containers, but when people use this word they are usually referring to the docker command itself. The company meanwhile, is having a bit of a tough time lately.
Kubernetes
Possibly the biggest buzzword there is right now, Kubernetes is an orchestration tool for containers. That means it manages the uptime of containers, where they are running, and how many of them are running. Then it does a bunch of other stuff like load balance services into and out of containers, manage storage, etc. You can think of it as a distributed operating system — some of the biggest workloads in the world run on Kubernetes. It is quite hard to build it yourself into any meaningful production state. My personal opinion — give GKE a shot.
Shameless plug incoming, but for a great 101 on all of the above, see: https://acloud.guru/learn/gke-beginner-to-pro
The Cloud
How did I miss that one? I’m sure you know what this is though — other people’s servers for rent. Google, Amazon and Microsoft have much more money to spend on data-centres, hardware and anally-retentive cable management than you do.
Serverless
Wait, where did other people’s servers go? Of course, they are still there. They’ve just been abstracted away from you so you don’t have to manage them. Instead you’ll be dealing with one of the following managed services:
A service that lets you run your application on a platform, without worrying too much about servers. For example, Google’s original cloud product App Engine (now 10 years old!) let’s you deploy your Java, PHP, Python, Node or Golang application to the web without deploying or configuring servers.
Functions as a Service
A service that lets you deploy a much smaller application, typically a single function that should be event-driven. Do you want a piece of code to execute when an HTTP endpoint is hit, or a specific message is received in a queue, and then exit? Use a function.
Containers as a Service
Develop and deploy your application as a container, without worrying about any of the infrastructure required to orchestrate it. Check out Cloud Run.
Infrastructure as Code
This one is pretty fundamental to the whole movement — stop thinking in an operational or imperative way, performing tasks consecutively or building things manually. Be declarative. Describe your infrastructure in code, using software development tools and processes. Then automate the deployment of this code (see below) to build your infrastructure.
CI/CD
These are software development processes: Continuous Integration (CI) and Continuous Deployment (CD). They are primarily concerned with streamlining the development workflow in git so that you can rapidly but safely make changes and get them into production. This could be for an application, or for your infrastructure itself (now that you’ve written it as code). Atlassian has a good explanation of the concepts of CI/CD here. The biggest difference between the two is that CD is actually deploying changes to production continuously. This may sound scary but it’s how giants like Netflix manage their infrastructure and applications. CI/CD tools can be tricky to set up, but can make managing your overall stack much safer and more efficient in the long run.
But wait, there’s more…
There are a few hundred more things you may have heard of. The CNCF helpfully put them all on a terrifying interactive info-graphic, a bit like a wall of ancient runes designed to ward off trespassers who may wish to enter the sacred realm of cloud computing:
But the truth is, you really don’t need to know much about all of these things. You’ll need to know a moderate amount about a handful of them, and you’ll need to know a few of them quite well if they are chosen for your project. People talk about being “life-long learners” and that’s really the fundamental thing here. You have to enjoy learning, trying new things and adapting to change.
If you’re willing to embrace the approach of constantly learning new things, you can leap over that imaginary barrier and play with all these shiny toys with the rest of us. It’s like Christmas! Which means next year there will be a whole load of new toys to learn about…
