DevOps Garfield

Reference: K0s CSI Driver

First, CSI stands for Container Storage Interface. It is an API for storage drivers to interact with kubernetes ( k0s ). Storage drivers are software that facilitate data storage. However, different types of storage require different drivers to work. Some drivers are generic while others are specific to their vendor's features.

Note: This document assumes you already have an NFS server with at least 1 share. If you don't, see Red Hat's blog - Learning NFS through server and client configuration

The best way to install components and applications ( apps ) is with the helm app.

1. Download helm from the Helm website

   wget https://get.helm.sh/helm-v4.2.0-linux-amd64.tar.gz

   1. You should verify the download with the sha256sum command:

      sha256sum helm-v4.2.0-linux-amd64.tar.gz | grep 97dbeb971be4ac4b27e3839976d9564c0fb35c6f3b1da89dd1e292d236af4096

2. Extract the tarball:

   tar zxf helm-v4.2.0-linux-amd64.tar.gz

3. Move the file to a directory in your path

   sudo mv linux-amd64/helm /usr/local/bin/helm

4. Delete the extracted files and directories ( optional )

   rm -rf linux-amd64

5. Verify the helm command works

   helm version

1. Add the helm repo to your system

   # Add the official Kubernetes CSI Helm repository
   helm repo add csi-driver-nfs https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
   helm repo update

2. Install the driver with the custom k0s path

   # Install the driver with the customized k0s path
   sudo helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs \
     --namespace kube-system \
     --set kubeletDir=/var/lib/k0s/kubelet \
     --kubeconfig /var/lib/k0s/pki/admin.conf

3. Watch the pods deployment for about 3 minutes.

   watch -n1 'k0s kubectl --namespace=kube-system get pods --selector="app.kubernetes.io/instance=csi-driver-nfs"'

1. Create a YAML file to define a StorageClass and toggle dynamic provisioning. Save the following manifest as nfs-storageclass.yaml. Make sure to substitute server with your actual NFS host IP and share with your exported path

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: nfs-csi
   provisioner: nfs.csi.k8s.io
   parameters:
     server: 192.168.1.50 # Replace with your NFS Server IP
     share: /volume1/k8s  # Replace with your exported NFS path
   reclaimPolicy: Delete
   volumeBindingMode: Immediate
   mountOptions:
     - nfsvers=3
     - hard   # Pods wait if the NFS server goes offline
     - nolock # Use '-o nolock' to keep locks local

2. Create a PVC ( Persistent Volume Claim ) YAML file that makes the share available to the pods

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: test-nfs-pvc
   spec:
     accessModes:
       - ReadWriteMany
     storageClassName: nfs-csi
     resources:
       requests:
         storage: 5Gi

3. Apply the config to your cluster

   sudo k0s kubctl apply -f manifest-nfs-pvc.yaml

4. Check your PVC. You should see “Bound” in the STATUS column

   sudo k0s kubectl get pvc

1. Create a PV YAML file WITHOUT a storage class

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: existing-nfs-pv
   spec:
     capacity:
       storage: 100Gi # Match or exceed what your pods need
     volumeMode: Filesystem
     accessModes:
       - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain # Retain ensures data isn't deleted if the PVC drops
     storageClassName: "" # Must be blank or use a custom name to skip automatic provisioning
     csi:
       driver: nfs.csi.k8s.io
       volumeHandle: existing-nfs-shared-volume # Must be a unique string in the cluster
       volumeAttributes:
         server: 192.168.1.50 # Replace with your NFS Server IP
         share: /volume1/existing-data # Replace with your exact existing path
     mountOptions:
       - nfsvers=4.1
       - hard
       - nolock # Kept to prevent the rpc.statd error

2. Create the PV

   sudo k0s kubectl aply -f manifest-existing-nfs-pv.yaml

3. Check the PV ( optional )

   sudo k0s kubectl get pv

4. Create a PVC YAML file

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: existing-nfs-pvc
   spec:
     accessModes:
       - ReadWriteMany
     storageClassName: "" # Must remain blank to match the PV's blank storageClassName
     volumeName: existing-nfs-pv # Forces binding to your specific PV above
     resources:
       requests:
         storage: 100Gi # Must be less than or equal to the PV size

5. Check the PVC ( optional ). You should see “Bound” in the STATUS colunn

   sudo k0s kubectl get pvc

6. Start a web server pod that mounts the nfs share to verify access ( optional )
   1. Create an app YAML file

      apiVersion: v1
      kind: Pod
      metadata:
        name: nfs-app-pod
        labels:           # <--- ADD THIS BLOCK SO THE SERVICE CAN FIND IT
          name: nfs-app-pod
      
      spec:
        containers:
          - name: web-server
            image: nginx:latest
            volumeMounts:
              - name: nfs-storage
                mountPath: /usr/share/nginx/html # Path inside the container where data appears
        volumes:
          - name: nfs-storage
            persistentVolumeClaim:
              claimName: existing-nfs-pvc # Must exactly match your PVC name

   2. Deploy the app

      sudo k0s kubectl apply -f manifest-app-nfs-test.yaml

   3. Get a shell inside the pod to look at the file system and verify the files exist

      sudo k0s kubectl exec -it nfs-app-pod -- bash
      ls -lh /usr/share/nginx/html

1. Install MetalLB loadbalancer

   helm repo add metallb https://metallb.github.io/metallb
   helm repo update
   helm install metallb metallb/metallb \
     --namespace metallb-system \
     --create-namespace \
     --kubeconfig /var/lib/k0s/pki/admin.conf

2. Create an MetalLB YAML file to setup an IP pool/range

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     name: local-pool
     namespace: metallb-system
   spec:
     addresses:
       - 192.168.1.200-192.168.1.210 # <--- Change to your free network IP range
   ---
   apiVersion: metallb.io/v1beta1
   kind: L2Advertisement
   metadata:
     name: local-advertisement
     namespace: metallb-system
   spec:
     ipAddressPools:
       - local-pool

3. Create an app service YAML file to expose the app outside the cluster

   apiVersion: v1
   kind: Service
   metadata:
     name: nfs-web-service
   spec:
     type: LoadBalancer
     selector:
       name: nfs-app-pod # MUST match the label on your Pod
     ports:
       - protocol: TCP
         port: 80        # The port open to your network
         targetPort: 80  # The port Nginx is listening on inside the container

4. Apply the config to your cluster

   sudo k0s kubctl apply -f manifest-app-nfs-web-service-lb.yaml