Reference: Bootstrapping clusters with kubeadm

1. System prep ( RPM-based distros )
   1. Install packages:

      sudo dnf install nc jq socat iproute-tc -y

   2. Open firewall ports:

      sudo firewall-cmd --add-port=6443/tcp --permanent && sudo firewall-cmd --reload && sudo firewall-cmd --list-all
      sudo firewall-cmd --add-port=10250/tcp --permanent && sudo firewall-cmd --reload && sudo firewall-cmd --list-all

   3. Disable swap as containers can't use it:

      sudo swapoff -a

   4. Edit fstab file to permanently disable swap by adding the “#” character to the beginning of the line:

      sudo vim /etc/fstab
      # /dev/mapper/<hostname>-swap none                    swap    defaults        0 0

   5. Enable IPv4 forwarding:

      cat << EOF | sudo tee /etc/sysctl.d/k8s.conf
      net.ipv4.ip_forward = 1
      EOF
      sudo sysctl --system

2. Set SELinux in permissive mode (effectively disabling it):

   sudo setenforce 0
   sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

3. enable netfilter system module:

   sudo modprobe br_netfilter

4. Add the following contents to /etc/yum.repos.d/kubernetes.repo to setup RPM repos:

   [kubernetes]
   name=Kubernetes
   baseurl=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/
   enabled=1
   gpgcheck=1
   gpgkey=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/repodata/repomd.xml.key
   exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni

5. Add the following contents to /etc/yum.repos.d/cri-o.repo to setup RPM repos:

   [cri-o]
   name=CRI-O
   baseurl=https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.31/rpm/
   enabled=1
   gpgcheck=1
   gpgkey=https://pkgs.k8s.io/addons:/cri-o:/stable:/v1.31/rpm/repodata/repomd.xml.key

6. Install the k8s packages:

   sudo dnf install -y cri-o container-selinux kubelet kubeadm kubectl --disableexcludes=kubernetes

7. Start the cri-o service:

   sudo systemctl enable --now crio.service

8. Start the kublet service:

   sudo systemctl enable --now kubelet.service

1. Initialize kubeadm with a specified pod network in CIDR format. The one shown here is common. To have a true H/A cluster, create an external DNS entery that maps to this control plane node's IP address. Later, you can create a load balancer with the control plane nodes' IPs and update the DNS entry.
   
   If you choose not to have multiple controle plane nodes, then don't pass in the endpoint option. Turning a single control plane cluster created without –control-plane-endpoint into a highly available cluster is not supported by kubeadm.

   sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=<dns.name>

2. Copy the join command from the output of the above init command and store it somewhere. It will look something like this.

   !!!! DO NOT COPY THIS COMMAND FROM HERE !!!!
   ============================================
   kubeadm join 192.168.1.17:6443 --token 8gf1ah.7boas234f8a663gas \
     --discovery-token-ca-cert-hash sha256:44f76a2d10922b7ac980faebcd42ae75f061b6cf4c5ccacef8937d0f064c

You should now have a 1-box k8s cluster as a control plane node. By copying the above command, you prepared for adding more nodes to your cluster.

1. Setup non-root user account as k8s admin:

   mkdir -p $HOME/.kube
   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown $(id -u):$(id -g) $HOME/.kube/config

1. Download the flannel networking manifest for the Kubernetes API datastore:

   curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.2/manifests/canal.yaml -O

2. Install Canal, which contains both Calico ( pod networking policies ) and flannel ( for pod networking ):

   kubectl apply -f canal.yaml

If you want a highly avaiable cluster, meaning you want multiple control plane nodes, then you must create additional servers and join them to the cluster as control plane nodes. The kubeadm command has special paramenters to designate the new node as such.

If you want only 1 control plane node, but still want multiple worker nodes, skip to the next section.

1. Run the system prep process in steps 1-8.
   
   
2. Assuming you created a DNS record for your control plane endpoint and added the --control-plane-endpoint=<dns.name> to your kubeadm init command, run the following command:

   !!!! DO NOT COPY THIS COMMAND FROM HERE !!!!
   ============================================
   kubeadm join k8s01.home.mygarfield.us:6443 --token 1vk4n8.bgys7j3f2348cad42y4v \
     --discovery-token-ca-cert-hash sha256:ac167419d422b46ad7182349fda14f72e7b7745fc009f2dd2db97b7c6 \
     --control-plane

3. To verify the new node has been added to the cluster, run the following command:

   kubectl get nodes

In most cases, you want to have multiple worker nodes. This is where your containers/applications will run.

1. Run the system prep process in steps 1-8
   
   
2. Run the join command copied from above as root ( I recommend you run with sudo as in sudo join… )

1. Run the following command:

   kubectl get nodes

It should look something like this

[garfield@k8s01 ~]$ kubectl get nodes
NAME                       STATUS   ROLES           AGE   VERSION
k8s01.home.mygarfield.us   Ready    control-plane   17h   v1.31.1
k8s02.home.mygarfield.us   Ready    <none>          10m   v1.31.1
k8s03.home.mygarfield.us   Ready    <none>          6s    v1.31.1

Helm is a package manager for Kubernetes. A helm chart is a collection of text files that describe how to install an application.

1. Download helm:

   wget https://github.com/helm/helm/releases

2. Uncompress the package and install the binary. This will create a new directory with the name of your system architecture (eg. linux-amd64).

   tar zxf helm-*.tar.gz

   1. Inside, there will be the helm binary and 2 text files.

      ls linux-amd64/
      helm  LICENSE  README.md

3. Copy the binary to a location inside your path.

   sudo cp linux-amd64/helm /usr/local/bin/helm

4. Verify the helm command works.
   You should get comething like: version.BuildInfo{Version:“v3.16.1”, GitCommit:“5a5449dc42be07001fd5771d56429132984ab3ab”, GitTreeState:“clean”, GoVersion:“go1.22.7”}

   helm version