We have opted for K3s, a lightweight Kubernetes distribution, for our setup. This decision is driven by K3s' ability to deliver a full-fledged Kubernetes experience while demanding lesser resources in terms of CPU and RAM compared to other distributions.
Here is our nodes' reference for this section:
- Node1 - 10.0.0.60 - Serves as both a Kubernetes controller and worker node
- Node2 - 10.0.0.61 - Serves as a Kubernetes worker node
- Node3 - 10.0.0.62 - Functions as a Kubernetes worker node and an NFS server
- Node4 - 10.0.0.63 - Acts as a Kubernetes worker node
Using your preferred SSH client, log into Node1. Execute the following command:
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 --disable servicelb --token myrandompassword --node-ip 10.0.0.60 --disable-cloud-controller --disable local-storage
Let's dissect this command:
- --write-kubeconfig-mode 644 - Specifies the permissions for the kubeconfig file. While optional, it's needed if you plan to later connect to the Rancher manager.
- --disable servicelb - Disables the service load balancer. We will use Metallb instead.
- --token - Defines the token used to connect to the K3s master node. Choose a strong random password and keep it safe.
- --node-ip - Binds the K3s master node to a specific IP address.
- --disable-cloud-controller - Turns off the K3s cloud controller, which we don't need for this setup.
- --disable local-storage - Deactivates the K3s local storage as we'll set up Longhorn and NFS as storage providers instead.
Post-installation, verify the installation success with
kubectl get nodes, which should return an output similar to:
root@cube01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION cube01 Ready control-plane,master 6m22s v1.25.6+k3s1
Next, add the worker nodes. SSH into Node2 to Node4 and run the following command:
curl -sfL https://get.k3s.io | K3S_URL=https://10.0.0.60:6443 K3S_TOKEN=myrandompassword sh -
You can run this command simultaneously on all nodes. After the script completes, execute
kubectl get nodes on Node1. This should yield:
root@cube01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION cube01 Ready control-plane,master 43m v1.25.6+k3s1 cube04 Ready <none> 38s v1.25.6+k3s1 cube02 Ready <none> 35s v1.25.6+k3s1 cube03 Ready <none> 34s v1.25.6+k3s1
Although optional, labeling nodes is recommended to represent the "worker" role instead of "". Labels are more than cosmetic; they help specify a node for running certain workloads. For example, a specialized hardware node, like a Jetson Nano, can be labeled, and applications that require
such hardware can be directed to run exclusively on that node.
Label your nodes with the key:value pair kubernetes.io/role=worker to get cleaner output from
kubectl get nodes.
kubectl label nodes cube01 kubernetes.io/role=worker kubectl label nodes cube02 kubernetes.io/role=worker kubectl label nodes cube03 kubernetes.io/role=worker kubectl label nodes cube04 kubernetes.io/role=worker
Another useful label is to denote the node-type as "worker" for deploying applications. "node-type" is an arbitrary key name and can be changed as needed.
kubectl label nodes cube01 node-type=worker kubectl label nodes cube02 node-type=worker kubectl label nodes cube03 node-type=worker kubectl label nodes cube04 node-type=worker
If your Node4 is a Nvidia Jetson, you might label it as node-type=jetson, allowing specific workloads (like ML containers) to run exclusively on that node.
After labeling, your nodes should resemble:
root@cube01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION cube01 Ready control-plane,master,worker 52m v1.25.6+k3s1 cube02 Ready worker 10m v1.25.6+k3s1 cube03 Ready worker 10m v1.25.6+k3s1 cube04 Ready worker 10m v1.25.6+k3s1
To list all labels per node, use:
kubectl get nodes --show-labels
Well done! You've successfully set up a Kubernetes cluster. However, there are additional steps to improve the cluster's functionality. Continue reading our guide for these next steps.
Updated 5 months ago