Beginner's Guide on Terraform and Kubernetes: Your First k3d Cluster

Terraform and Kubernetes are cool. But I don’t see a lot of tutorials that are easy enough for beginners and full-stack data scientists. Kubernetes isn’t supposed to be difficult. Still, I find most of the tutorials too difficult for beginners. Do you need to know about DaemonSet before you deploy your app? What about Horizontal Pod Autoscaler ?

I don’t believe you need to know all of those.

I’ll try to do something different here. Terraform, Kubernetes, and DevOps that you can actually see them happening. I will take few shortcuts and use sane defaults. The goal is to know “just enough” to deploy and maintain a cluster.

In this series, you’ll have a chance to …

• Deploy a data visualization application on Kubernetes using Terraform in 10 minutes
• Set up a Kubernetes cluster on AWS EKS
• Deploy your first AWS Lambda function
• Deploy a database on Kubernetes and configure it to handle a data stream
• … and many more …

All in Terraform 👍

Today, I’ll show you how to spin up your first Kubernetes cluster on your local computer. You don’t want to start your Kubernetes journey by paying Amazon (or other cloud provider).

But before we do anything. Let’s make sure that we understand why we (might) need Kubernetes.

🤔 Do you need Kubernetes?

You first read about Kubernetes, Terraform, Infrastructure as a Code and you started thinking “Wow! This is going to make my life easier”. But after you tried few examples, you started to wonder if this whole DevOps thing is necessary. You asked “Can I just deploy my app on Heroku?”

My answer: PaaS if you can. Kubernetes if you must.

Why? Should I be promoting Kubernetes and Terraform here?

People can keep saying how easy Kubernetes is all day long. But working with k8s will never be easier than working with PaaS platform like Heroku.

Still, in some deployments, it’s easier to use k8s. For example, you might work with an application that needs to run a memory-intensive task nightly. How would you configure PaaS to handle that spike?

This is where Kubernetes really shines. K8s is great if you want to manage resources.

Now, let’s start spinning up a local cluster so you can actually see Kubernetes working.

💻 Step 1: Install the tools you need

You will need …

• Docker on your local machine
• Terraform binary
• k3d to create a cluster on your local machine
• Kubernetes tools below to interact with the cluster

Why k3d?

You probably think that you need to pay AWS or Google to try Kubernetes. No, you don’t. You can spin up a cluster and play with it on your local computer. k3d is a lightweight tool to emulate the real Kubernetes cluster. I find this to be easier to set up and lighter than minikube or Kubernetes on Docker Desktop. I use k3d to test my deployment before deploying to the cloud all the time.

Kubernetes tools

I’m not a big fan of too many tools. So I curated few tools that will make your K8s life easier.

🛥️ kubectl https://kubernetes.io/docs/reference/kubectl/overview/

You can’t escape kubectl if you want to use K8s. This is a must-have if you want to interact with the cluster directly from your terminal. Everyone use it and so all the tutorials assume that you have it on your computer.

🐶 K9s: https://k9scli.io

You don’t want to remember all the kubectl commands. Luckily, you can run K9s and navigate your cluster from the terminal interface. You can download a binary from their GitHub or run brew install derailed/k9s/k9s if you are using MacOS.

⚓ Helm: https://helm.sh

Helm is a de-facto standard to deploy a pre-packaged application on a Kubernetes cluster. Imagine you want a Postgres database on your cluster, you can just set tell Helm to install a pre-defined Postgres formula (aka. “Helm chart”) and you will get a database running. To install Helm on your local computer, you can download a binary from their GitHub or run brew install helm if you are using MacOS.

🚀 Spin up your first cluster

Ok. Enough installing. Let’s do something fun. Let’s create your first cluster on your local computer. Remember that, in production, you’ll probably do this step on one of your cloud providers (hopefully with Terraform). There will be more things to consider such as networking, autoscaling, monitoring, and logging. But we’ll leave them to other times.

So let’s create a cluster by running this command:

1

 k3d cluster create -p "8080:80@loadbalancer" mycluster

In just few seconds, you’ll have a new cluster called mycluster

You can examine the cluster by starting k9s and this is what you’ll see:

Wow. Kubernetes is so easy, right? Well, kinda. But the fun part is after this. We’re going to deploy an app with Terraform and learn about Kubernetes at the same time. Keep on reading.

🧹 Cleaning things up

You can stop your k3d cluster but I prefer a harsher way. I delete them when I’m done. Just like flushing the toilet 🚽, you can run this k3d command:

1

 k3d cluster delete mycluster

💥 Boom! Everything is gone.