Update Kubernetes certificates
Lars Jönsson, 2024-06-23
Information about how to replace expired certificates in a Kubernetes
node. Normally most of the certificates will be replaced automatically
in a running system and kubeadm can be also be used for for
replacing the certificates. When the Kubernetes cluster is not running
all the time, it may end up in a non-startable cluster and the
certificates needs to be updated manually.
This instruction also include information about how to replace the
HTTPS certificate for the Dashboard.
Kubernetes certificates
Most of the Kubernetes certificates can be renewed by using
kubeadm. First check they are still valid.
sudo kubeadm certs check-expirationUpdate the certificates if needed.
sudo kubeadm certs renew allUpdate the users Kubernetes configuration, if the admin.conf file is
updated.
cd ~/.kube
mv config old-config
sudo cp /etc/kubernetes/admin.conf config
sudo chown `id -un`:`id -gn` config Kubelet certificates
The kubelet certificate needs to be udated manually if the it has expired.
NOTE Check and update of
kubeletcertificates has do be done locally on all nodes, but it is easier to remove the non-controller nodes and re-join them
Check the current certificate
NOTE If the ca certificate (
pki/ca.crt) is updated, the kubelet certificates needs to be updated even if they are still valid.
Print the current kubelet certificate:
sudo su
cd /etc/kubernetes/
openssl x509 -noout -dates -in `grep client-certificate kubelet.conf | awk '{print $2}'`
exitA printout similar to this will be printed. Check that the end date is still valid.
notBefore=Dec 5 14:55:38 2022 GMT
notAfter=Dec 5 14:55:41 2023 GMTCreate and install new kubelet key and certificate
As a root, jump to the Kubernetes configuration folder.
sudo su
cd /etc/kubernetesCreate a new key and certificate:
openssl genrsa -out kubelet.key 2048
openssl req -new -key kubelet.key -subj /O=system:nodes/CN=system:node:k8s-master \
-out kubelet.csr -addext "keyUsage = critical, digitalSignature, keyEncipherment" \
-addext "extendedKeyUsage = TLS Web Client Authentication" \
-addext "basicConstraints = critical, CA:false"
openssl x509 -req -in kubelet.csr -CA /etc/kubernetes/pki/ca.crt \
-CAkey /etc/kubernetes/pki/ca.key -out kubelet.crt -days 300 -copy_extensions=copyallPut the certificate and key into the same file. The file name includes the current date and time.
DATETIME=`date +%F-%H-%M-%S`
cat kubelet.crt kubelet.key > /var/lib/kubelet/pki/kubelet-client-${DATETIME}.pemRotate the certificate by letting the symblic link point to the new file.
rm /var/lib/kubelet/pki/kubelet-client-current.pem
ln -s /var/lib/kubelet/pki/kubelet-client-${DATETIME}.pem /var/lib/kubelet/pki/kubelet-client-current.pemEnsure that the latest ca certificate is includede in the
certificate-authority-data section of the kubelet.conf file.
cert=`base64 -w 0 pki/ca.crt` bash -c 'sed -i.old \
"s/certificate-authority-data: .*/certificate-authority-data: $cert/" my-kubelet.conf'Restart the kubelet service and verify that it is running.
systemctl restart kubelet.service
systemctl status kubelet.serviceEnd the root user session:
exitKuberneters Dashboard certificate
The Kubernetes Dashboard HTTPS certificate is signed by a CA to
prevent Web browsers from complaining about invalid certificates. If
it is signed by a local CA, the root and intermediate certificates of
the CA needs to installed in the Web browser.
It is only the Kubernetes part of the certificate that is covered, not the signing process at the CA.
More information about certificates is available in the Certificates guide.
Generate a key:
$ openssl genrsa -aes256 -out 192.168.121.30-dashboard.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
....................................................................................................................................++++
....................................................................++++
e is 65537 (0x010001)
Enter pass phrase for 192.168.121.30-dashboard.key:
Verifying - Enter pass phrase for 192.168.121.30-dashboard.key:Generate the signing request:
$ openssl req -new -key 192.168.121.30-dashboard.key \
-addext "subjectAltName = IP:192.168.121.30" \
-out 192.168.121.30-dashboard.csr
Enter pass phrase for 192.168.121.30-dashboard.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:SE
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:MyCity
Organization Name (eg, company) [Default Company Ltd]:MyCompany
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:192.168.121.30
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:Copy the signing request to the CA:
$ scp 192.168.121.30-dashboard.csr <ca-server>:.Sign the CSR. It can be signed according these instructions if you have your own private CA.
Get the signed certificate from the CA:
$ scp <ca-server>:192.168.121.30-dashboard.crt .Remove the password from the key file:
$ openssl rsa -in 192.168.121.30-dashboard.key -out 192.168.121.30-dashboard-unencrypted.key
Enter pass phrase for 192.168.121.30-dashboard.key:
writing RSA keyStore the password free key and certificate at a location where it will be installed into the Kubernetes cluster:
$ cp 192.168.121.30-dashboard-unencrypted.key ~/certs/dashboard.key
$ cp 192.168.121.30-dashboard.crt ~/certs/dashboard.crtTo update the Kubernetes Dashboard certificate, it is easiest to just uninstall it and install it again with the new key and certificate. The procesure to install Kubernetes Dashboard is covered by a separate instruction called Install Kubernetes Dashboard.