The DevOps toolbox, mapped

At work you hear tool names all day: Docker, Terraform, Kubernetes, Datadog. Each one sounds like something you must learn tonight.

You do not. Each tool does one job, and there are only eight jobs. Learn the jobs first, and every new name has a place to go.

Sort tools by job, not by name

A job is a problem the team must solve. A tool is one product that solves it. Many products do the same job, and each team picks one.

┌────────────────────────────────────────────┐
│                                            │
│            job: run the tests              │
│                     │                      │
│        ┌────────────┼────────────┐         │
│        ▼            ▼            ▼         │
│   ┌─────────┐  ┌─────────┐  ┌─────────┐    │
│   │ GitHub  │  │ GitLab  │  │ Jenkins │    │
│   │ Actions │  │ CI      │  │         │    │
│   └─────────┘  └─────────┘  └─────────┘    │
│          same job, different brand         │
│                                            │
└────────────────────────────────────────────┘

When you hear a new name, ask one question: which job does it do? If the answer is "it runs the tests", it sits next to GitHub Actions. You know what it is for, even if you never saw it before.

Here are the eight jobs in three groups. The groups follow the loop from the first lesson: build, run, then watch.

┌─────────────────────────────────────────────┐
│                                             │
│   the devops toolbox                        │
│   ├─ build and ship                         │
│   │  ├─ 1 code             you use it now   │
│   │  ├─ 2 CI/CD            chapter 5        │
│   │  └─ 3 package          chapter 4        │
│   ├─ run                                    │
│   │  ├─ 4 cloud            chapter 3        │
│   │  ├─ 5 orchestrate      chapter 6        │
│   │  └─ 6 infra as code    chapter 7        │
│   └─ watch and protect                      │
│      ├─ 7 monitor          chapter 8        │
│      └─ 8 secure           chapter 9        │
│                                             │
└─────────────────────────────────────────────┘

Build and ship

These three jobs turn your code into something that can run. You already touch the first two every day.

• Code. Git records every change you make. GitHub stores the code and runs your pull requests. Together this is version control.
• CI/CD. CI/CD is the robot from the last two lessons. The checks on your pull request come from it. See chapter 5.
• Package. Docker puts your app, its Node.js version and every library into one container image. The image runs the same on your laptop and on a server. See chapter 4.

Run

These three jobs give your app a place to live and keep it alive.

• Cloud. A cloud provider rents you servers, databases and networks. You pay for what you use. Vercel runs your app on cloud servers and hides them from you. See chapter 3.
• Orchestrate. One container is easy to run. Fifty containers on ten servers need a manager. An orchestrator does that work. Kubernetes is the best-known one. See chapter 6.
• Infra as code. The team does not click buttons in the AWS console. It writes the servers down in a file, and Terraform creates them. The file lives in Git, so a pull request reviews every change. This is infrastructure as code. See chapter 7.

Watch and protect

These two jobs start after users arrive. They tell the team when something is wrong, and they keep attackers out.

• Monitor. Monitoring answers one question: is the app healthy right now? Prometheus collects numbers, such as requests per second. Grafana draws them on dashboards. Datadog does both as a paid service. See chapter 8.
• Secure. A secret must never sit in the code. HashiCorp Vault stores secrets and gives them to the app at start. Dependabot opens a pull request when a package you use has a known security hole. Trivy scans container images for the same holes. See chapter 9.

The map on a real team

Here is one change, followed through all eight jobs, on a team that runs its own servers on AWS:

Now compare your own Next.js app on Vercel. The jobs are the same. Vercel does many of them for you.

Vercel did not remove the jobs. It hides them. When a team needs a long-running server, a special database or a lower bill at scale, the jobs come back, and so do the tools.

Read more

• The CNCF Landscape. Hundreds of cloud tools, sorted by the job they do.
• What the heck is Backstage anyway? from Spotify. The story above.
• Kubernetes overview from kubernetes.io. What an orchestrator does, in one page.
• Docker in Action, 2nd edition, chapter 1: read 1.5 on Docker in the larger ecosystem.

The next lesson is "What is a server?". It opens the next chapter, "Servers and networks".