DevOps Garfield

Installation process

Reference: Bootstrapping clusters with kubeadm

System prep ( RPM-based distros )

Install packages:

sudo dnf install nc jq socat iproute-tc -y

Open firewall ports:

sudo firewall-cmd --add-port=6443/tcp --permanent && sudo firewall-cmd --reload && sudo firewall-cmd --list-all
sudo firewall-cmd --add-port=10250/tcp --permanent && sudo firewall-cmd --reload && sudo firewall-cmd --list-all

Disable swap as containers can't use it:
```
sudo swapoff -a
```
Edit fstab file to permanently disable swap by adding the “#” character to the beginning of the line:
```
sudo vim /etc/fstab
# /dev/mapper/<hostname>-swap none                    swap    defaults        0 0
```

Enable IPv4 forwarding:

cat << EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system

Set SELinux in permissive mode (effectively disabling it):

sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

enable netfilter system module:
```
sudo modprobe br_netfilter
```

Add the following contents to /etc/yum.repos.d/kubernetes.repo to setup RPM repos:

[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni

Add the following contents to /etc/yum.repos.d/cri-o.repo to setup RPM repos:

[cri-o]
name=CRI-O
baseurl=https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.31/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.31/rpm/repodata/repomd.xml.key

Install the k8s packages:

sudo dnf install -y cri-o container-selinux kubelet kubeadm kubectl --disableexcludes=kubernetes

Start the cri-o service:

sudo systemctl enable --now crio.service

Start the kublet service:

sudo systemctl enable --now kubelet.service

Initialize cluster process

Initialize kubeadm with a specified pod network in CIDR format. The one shown here is common. To have a true H/A cluster, create an external DNS entery that maps to this control plane node's IP address. Later, you can create a load balancer with the control plane nodes' IPs and update the DNS entry.

If you choose not to have multiple controle plane nodes, then don't pass in the endpoint option. Turning a single control plane cluster created without –control-plane-endpoint into a highly available cluster is not supported by kubeadm.
```
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=<dns.name>
```

Copy the join command from the output of the above init command and store it somewhere. It will look something like this.

!!!! DO NOT COPY THIS COMMAND FROM HERE !!!!
============================================
kubeadm join 192.168.1.17:6443 --token 8gf1ah.7boas234f8a663gas \
  --discovery-token-ca-cert-hash sha256:44f76a2d10922b7ac980faebcd42ae75f061b6cf4c5ccacef8937d0f064c

You should now have a 1-box k8s cluster as a control plane node. By copying the above command, you prepared for adding more nodes to your cluster.

User setup process

Setup non-root user account as k8s admin:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

The kubeconfig file admin.conf that kubeadm init generates contains a certificate with Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin. The group kubeadm:cluster-admins is bound to the built-in cluster-admin ClusterRole. Do not share the admin.conf file with anyone.

kubeadm init generates another kubeconfig file super-admin.conf that contains a certificate with Subject: O = system:masters, CN = kubernetes-super-admin. system:masters is a break-glass, super user group that bypasses the authorization layer (for example RBAC). Do not share the super-admin.conf file with anyone. It is recommended to move the file to a safe location.

See Generating kubeconfig files for additional users on how to use kubeadm kubeconfig user to generate kubeconfig files for additional users.

Pod network setup process

Download the flannel networking manifest for the Kubernetes API datastore:

curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.2/manifests/canal.yaml -O

Install Canal, which contains both Calico ( pod networking policies ) and flannel ( for pod networking ):
```
kubectl apply -f canal.yaml
```

Highly Available ( multi-node control plane ) cluster setup process

If you want a highly avaiable cluster, meaning you want multiple control plane nodes, then you must create additional servers and join them to the cluster as control plane nodes. The kubeadm command has special paramenters to designate the new node as such.

If you want only 1 control plane node, but still want multiple worker nodes, skip to the next section.

Run the system prep process in steps 1-8.

Assuming you created a DNS record for your control plane endpoint and added the --control-plane-endpoint=<dns.name> to your kubeadm init command, run the following command:

!!!! DO NOT COPY THIS COMMAND FROM HERE !!!!
============================================
kubeadm join k8s01.home.mygarfield.us:6443 --token 1vk4n8.bgys7j3f2348cad42y4v \
  --discovery-token-ca-cert-hash sha256:ac167419d422b46ad7182349fda14f72e7b7745fc009f2dd2db97b7c6 \
  --control-plane

To verify the new node has been added to the cluster, run the following command:
```
kubectl get nodes
```

Add worker nodes to the cluster setup

In most cases, you want to have multiple worker nodes. This is where your containers/applications will run.

Run the system prep process in steps 1-8
Run the join command copied from above as root ( I recommend you run with sudo as in sudo join… )

Verify worker nodes have successfully joined the cluster

Run the following command:
```
kubectl get nodes
```

It should look something like this

[garfield@k8s01 ~]$ kubectl get nodes
NAME                       STATUS   ROLES           AGE   VERSION
k8s01.home.mygarfield.us   Ready    control-plane   17h   v1.31.1
k8s02.home.mygarfield.us   Ready    <none>          10m   v1.31.1
k8s03.home.mygarfield.us   Ready    <none>          6s    v1.31.1

Install applications in your cluster

Helm is a package manager for Kubernetes. A helm chart is a collection of text files that describe how to install an application.

Download helm:

wget https://github.com/helm/helm/releases

Uncompress the package and install the binary. This will create a new directory with the name of your system architecture (eg. linux-amd64).
```
tar zxf helm-*.tar.gz
```
1. Inside, there will be the helm binary and 2 text files.
```
ls linux-amd64/
helm  LICENSE  README.md
```
Copy the binary to a location inside your path.
```
sudo cp linux-amd64/helm /usr/local/bin/helm
```
Verify the helm command works.
You should get comething like: version.BuildInfo{Version:“v3.16.1”, GitCommit:“5a5449dc42be07001fd5771d56429132984ab3ab”, GitTreeState:“clean”, GoVersion:“go1.22.7”}
```
helm version
```

Table of Contents