ESXi 6.7 web UI SSL certificate algorithm

I needed to rotate the certificates on a standalone VMware ESXi 6.7 host today, it seemed like a good opportunity to bring the certificate up-to-date a bit, so I duly created an ECC (elliptical curve) key and had the certificate signed.

I was unable to install the certificate through the UI, instead receiving an unspecific "Cannot change the host configuration", and an instruction to read an unspecific log file (none of them seemed to log anything related).  But it seems like a common enough problem, and simply something that doesn't work very well in the UI.

Instead you can update the individual parts directly via SSH:

/etc/vmware/ssl/rui.key - contains an unencrypted private key in Base64 PEM format
/etc/vmware/ssl/rui.crt - contains a public certificate in Base64 PEM format

You can then restart the host, or just the management agents.  And oops, everything is dead.

Something seemed to object to the new certificate and my copy in the ssl directory vanished over a reboot, so no rollback available.  You can though get yourself out of trouble by generating new self-signed certificates via this command:

/sbin/generate-certificates

This gets the Web UI back up and running.  It turns out, that ESXi does not support ECC keys, despite it being fairly standard for a number of years.  Instead you have to fall back to slow RSA keys.  I created a new key and new signed certificate and installed it in the same way via SSH (as even with the correct format, the UI doesn't work), and everything was fine - and no more security warnings.

I see so many VMware setups without valid certificates in place, and so many "sysadmins" blindly click past the browser warnings.  This bad behaviour makes a MITM attack against your management hosts easy as admins who ignore TLS warnings won't spot that the certificate has changed to that of one belong to a malicious actor.  Security 101 folks.