MySQL/MariaDb failing to start after OpenSUSE update

There seemed to be a fairly large update to openSUSE Leap 15.2 that I recently ran on one of my development vms.  It all seemed to go ok apart from MariaDB (drop in alternative to MySQL).  The systemd startup script hung for a while before failing.  On investigating a bit, the error was:

And of course, hunting through the systemd or MySQL log locations didn't tell me anything useful.  Back to basics, work out what's going on and isolate the problem.  Starting up a systemd service uses a service file that is usually found in /usr/lib/systemd/system, in this case, mariadb.service.  The service file referenced a script /usr/lib/mysql/mysql-systemd-helper upgrade.  This is a shell script, so can easily be read in a text-editor, and navigating through the script quickly found the error detailed above, which happened immediately after a call to start the database in a protected/single-user-esq mode for running upgrades, with a randomly named unix socket to connect to (which can be obtained through ps).

The next step was a command called "mysql_wait", not something I'd encountered before, but this turned out to be a routine in the script to check the server for a while to see if it's accepting connections.

Armed with the socket file, and the detail of this subroutine, I tried to connect up manually by running the same command whilst waiting for the service to come online:

mysqladmin --socket=/var/tmp/mysql-protected.XXXX/mysql.sock ping

Unlike the script which sent all output into oblivion, I could see any errors, and immediately my error was returned.  And unrecognised statement in my config (/etc/my.ini), I removed this now obsolete statement from my config (which hadn't been updated, natch) and retried the service and it started ok.

TLDR;

Well there isn't a quick answer to this one, it's how to troubleshoot the problem.

1. Whilst the service is waiting to start, run ps axu | grep mysql to get the temporary socket path
2. Attempt to use mysqladmin ping along with the temporary socket and look for the error

You can also start mysql directly to give yourself more time to figure out the problem, but you run the risk of services connecting to the database and your data getting corrupted, or lost if you need to rollback.  It's best to check your database is all fine before letting services connect.

Update: Logs

Thanks to Stefan Wimmel who contacted me to say that there is a log file that gets created in /var/tmp/mysql-protected.XXXX/log_update_run, which should help narrow down the problem.