When you followed another article on this wiki about installing a 3 Node cluster from scratch, you may have wondered how to add worker Nodes, if more performance is needed or if a Node especially for testing is needed. For example if you want to test NodeSelectors on different Node types.
This article will explain how to prepare a VM in Proxmox, create a bootable OS image and how to add the Node to the existing cluster.
First up the new Proxmox VM has to be created, which will the new Node for the current 3 Node cluster.
It is assumed, that prior knowledge on the basics of creating a Proxmox VM are present. This is why only the most important topics will be explained in further detail.
The VM should match all the following criteria:
After the VM is successfully created, we can start to create the custom OS image for the new Node(s).
We now will create a yaml configuration file, which is very similar to the ones used in the installation of the 3 Node OKD cluster. This official documentation shows a good template to use. The template below has been modified by me to include settings for DNS and routes for my gateway.
hosts:
- hostname: okd-worker01.okd.johannesmauser.de #Name of the Node
rootDeviceHints:
deviceName: /dev/sda # Name of the disks on the VM, most times for Proxmox it will be sda
interfaces:
- macAddress: BC:24:11:E3:33:E3 # MAC-Adresse of the VM
name: ens18 # Name of the network interface on the VM
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:E3:33:E3
ipv4:
enabled: true
address:
- ip: 10.1.0.21 # IP of the Node, has to be unique
prefix-length: 24 # Subnetmask
dhcp: false
dns-resolver:
config:
server:
- 1.1.1.1 # 1. DNS Server
- 8.8.8.8 # 2. DNS Server
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 10.1.0.1 # Gateway of the network
next-hop-interface: ens18
table-id: 254
The command to create the image expects the filename “nodes-config.yaml”.
Prior to creating the OS image, you need to log in to the OKD cluster via oc.
In the directory of the yaml config file the following command needs to be executed to start the creation of the OS image: oc adm node-image create
Creating the image will take approximately 5 to 10 minutes. After it finished the image will be located in the same directory with the name node.x86_64.iso.
Now the bootable image is present, the next step is to upload the image to Proxmox. Please navigate to the datastore called “local” on the bottom left on your Proxmox web interface. Next click on “ISO Images” and upload the image.
After the upload finished click on your Proxmox VM for the new Node, navigate to “Hardware” and click on “CD/DVD Drive”. Please activate the option to use an ISO image and select the image, which was just uploaded to the datastore “local”.
Please notice, that the boot order of the VM needs to have the disk as the first option and the CD/DVD drive as the second option, otherwise the VM won't be able to boot from the disk after writing the image to it and will get stuck.
After that the VM can now be started and the initial installation begins automatically. This will take about 15 to 30 minutes, afterwards the VM will reboot. After this restart the final image will be written to the disk and the VM will boot from the disk and not the CD/DVD drive.
Now the new Node(s) will be visible using the CLI or the webconsole. For the Node to fully join the cluster CertificateSigningRequests have to be approved. These CSRs can be approved via the webconsole by going to the Node tab, clicking on the blue status “discovered” and then choosing “approve”. The CSRs also can be approved via the CLI by running the command oc adm certificate approve [NAME_DES_CSR].
After the first approval, 2 more CSRs will automatically be approved. After a short time a final CSR will appear and needs to be manually approved. You can look for new CSRs in the GUI or via this command: oc get certificatesigningrequests.certificates.k8s.io -A.
This is what the output of the command above will look like when the last CSR's still waiting for approvement:
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
csr-7kb9v 7m48s kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-g7nns 2m45s kubernetes.io/kube-apiserver-client system:node:okd-worker01.okd.johannesmauser.de 24h Approved,Issued
csr-gchnf 3m46s kubernetes.io/kubelet-serving system:node:okd-worker01.okd.johannesmauser.de <none> Pending
csr-ldgl2 2m52s kubernetes.io/kube-apiserver-client system:node:okd-worker01.okd.johannesmauser.de 24h Approved,Issued
Now after all the CSRs are approved and no errors remain, the Node will be visible and ready.
oc adm node-image create command.