Strange problem on qemu VPS

nomorewindows · 2020-09-28 03:53:38

Running a VPS with Arch Linux on it.

But something happened it was rebooted the other day for some reason by the provider I'm guessing.

It comes up according to the logs on it, but 100% packet loss when pinging it.

When I use the rescue mode I find the logs showing where it was booted, but no network connectivity from outside.

Can only access the ssh in rescue mode, and mount my partitions, but I don't think anything is out of the ordinary that I can find.

Have been running it for a couple of years.

No known problems with recent kernels that I know of. But apparently two have been pushed out in the last week.

fukawi2 · 2020-09-28 04:37:17

Does the provider have a console you can access?

nomorewindows · 2020-09-28 19:00:16

fukawi2 wrote:

Does the provider have a console you can access?

It uses java which isn't being enabled by browsers these days.

nomorewindows · 2020-09-28 19:39:34

They have a panel VNC function.

I've gotten in now.

But the network interface has decided to not come up for some reason.

nomorewindows · 2020-09-28 19:51:19

Brought the network interface up and now it's accessible from the internet.

But something hosed my mariadb database for wordpress.

nomorewindows · 2020-09-28 20:00:52

Mariadb hasn't been updated since Aug 2020, so it's not a new update to mariadb.

â mariadb.service - MariaDB 10.5.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Mon 2020-09-28 19:45:27 UTC; 11min ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
    Process: 431 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
    Process: 432 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
    Process: 441 ExecStart=/usr/bin/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
   Main PID: 441 (code=exited, status=1/FAILURE)
     Status: "MariaDB server is down"

Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.4.7.
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Starting shutdown...
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Plugin 'InnoDB' init function returned error.
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Unknown/unsupported storage engine: InnoDB
Sep 28 19:45:27  mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Aborting
Sep 28 19:45:27  systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 19:45:27  systemd[1]: mariadb.service: Failed with result 'exit-code'.
Sep 28 19:45:27  systemd[1]: Failed to start MariaDB 10.5.5 database server.

fukawi2 · 2020-09-29 00:46:09

Stop replying to your own thread when no-one else has - there's an edit button for a reason.

You can see the full logs with `journalctl -b -u mariadb.service` and that should show you the reason for the failure.

nomorewindows · 2020-09-29 14:01:17

fukawi2 wrote:

Stop replying to your own thread when no-one else has - there's an edit button for a reason.
You can see the full logs with `journalctl -b -u mariadb.service` and that should show you the reason for the failure.

-- Logs begin at Fri 2018-07-20 19:49:41 UTC, end at Tue 2020-09-29 13:59:02 UTC. --
Sep 28 19:26:23   systemd[1]: Starting MariaDB 10.5.5 database server...
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Note] /usr/bin/mariadbd (mysqld 10.5.5-MariaDB) starting as process 251 ...
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Warning] Could not increase number of max_open_files to more than 16384 (request: 32183)
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Warning] WSREP: Failed to guess base node address. Set it explicitly via wsrep_node_address.
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Warning] WSREP: Failed to guess base node address. Set it explicitly via wsrep_node_address.
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Warning] WSREP: Guessing address for incoming client connections failed. Try setting wsrep_node_incoming_address explicitly.
Sep 28 19:26:28   mariadbd[251]: 2020-09-28 19:26:28 0 [Note] WSREP: Node addr:
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Using Linux native AIO
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Uses event mutexes
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Number of pools: 1
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Completed initialization of buffer pool
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.4.7.
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [Note] InnoDB: Starting shutdown...
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] Plugin 'InnoDB' init function returned error.
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] Unknown/unsupported storage engine: InnoDB
Sep 28 19:26:29   mariadbd[251]: 2020-09-28 19:26:29 0 [ERROR] Aborting
Sep 28 19:26:29   systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 19:26:29   systemd[1]: mariadb.service: Failed with result 'exit-code'.
Sep 28 19:26:29   systemd[1]: Failed to start MariaDB 10.5.5 database server.
Sep 28 19:45:10   systemd[1]: Starting MariaDB 10.5.5 database server...
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] /usr/bin/mariadbd (mysqld 10.5.5-MariaDB) starting as process 420 ...
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Warning] Could not increase number of max_open_files to more than 16384 (request: 32183)
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Using Linux native AIO
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Uses event mutexes
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Number of pools: 1
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Completed initialization of buffer pool
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.4.7.
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [Note] InnoDB: Starting shutdown...
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] Plugin 'InnoDB' init function returned error.
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] Unknown/unsupported storage engine: InnoDB
Sep 28 19:45:11   mariadbd[420]: 2020-09-28 19:45:11 0 [ERROR] Aborting
Sep 28 19:45:11   systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 19:45:11   systemd[1]: mariadb.service: Failed with result 'exit-code'.
Sep 28 19:45:11   systemd[1]: Failed to start MariaDB 10.5.5 database server.
Sep 28 19:45:26   systemd[1]: Starting MariaDB 10.5.5 database server...
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] /usr/bin/mariadbd (mysqld 10.5.5-MariaDB) starting as process 441 ...
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Warning] Could not increase number of max_open_files to more than 16384 (request: 32183)
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Using Linux native AIO
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Uses event mutexes
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Number of pools: 1
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Completed initialization of buffer pool
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.4.7.
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [Note] InnoDB: Starting shutdown...
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Plugin 'InnoDB' init function returned error.
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Unknown/unsupported storage engine: InnoDB
Sep 28 19:45:27   mariadbd[441]: 2020-09-28 19:45:27 0 [ERROR] Aborting
Sep 28 19:45:27   systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 19:45:27   systemd[1]: mariadb.service: Failed with result 'exit-code'.
Sep 28 19:45:27   systemd[1]: Failed to start MariaDB 10.5.5 database server.

nomorewindows · 2020-09-29 14:16:20

Looking at https://jira.mariadb.org/browse/MDEV-18532

It said to remove the ib_logfile* from /var/lib/mysqld

Some errors were reported after doing that, but it looks like I have it running mysqld now.

Sep 29 14:13:14   mariadbd[723]: 2020-09-29 14:13:14 3 [ERROR] InnoDB: Page [page id: space=101, page number=7] log sequence number 424912371 is in the future! Current system log sequence number 295402520.
Sep 29 14:13:14   mariadbd[723]: 2020-09-29 14:13:14 3 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
Sep 29 14:13:14   mariadbd[723]: 2020-09-29 14:13:14 3 [ERROR] InnoDB: Page [page id: space=101, page number=19] log sequence number 424912291 is in the future! Current system log sequence number 295402520.
Sep 29 14:13:14   mariadbd[723]: 2020-09-29 14:13:14 3 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
Sep 29 14:13:16   mariadbd[723]: 2020-09-29 14:13:16 4 [ERROR] InnoDB: Page [page id: space=101, page number=8] log sequence number 413273291 is in the future! Current system log sequence number 295444796.
Sep 29 14:13:16   mariadbd[723]: 2020-09-29 14:13:16 4 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
Sep 29 14:13:16   mariadbd[723]: 2020-09-29 14:13:16 3 [ERROR] InnoDB: Page [page id: space=63, page number=79] log sequence number 330066530 is in the future! Current system log sequence number 295445950.
Sep 29 14:13:16   mariadbd[723]: 2020-09-29 14:13:16 3 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
Sep 29 14:13:18   mariadbd[723]: 2020-09-29 14:13:18 4 [ERROR] InnoDB: Page [page id: space=101, page number=17] log sequence number 338979625 is in the future! Current system log sequence number 295528081.
Sep 29 14:13:18   mariadbd[723]: 2020-09-29 14:13:18 4 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.

nomorewindows · 2020-09-29 14:27:05

Still not sure what to check about the network interface not coming up on its' own.

It is getting its' address via static dhcp from the hosting company.

Sometimes my home machines do this and after a couple of reboots it can sometimes link up on its' own. Otherwise, I have to manually restart my network interfaces.

If it doesn't all the servers of course don't start and have to be restarted manually.

I did restart the VPS several times without it linking on its' own.

Maybe this is a kernel or systemd bug?

Since the module loads, and just needs to be brought up with ifconfig, it is probably a problem with systemd.

Last edited by nomorewindows (2020-09-29 19:20:49)

loqs · 2020-09-29 14:41:21

fukawi2 wrote:

Stop replying to your own thread when no-one else has - there's an edit button for a reason.

You even quoted it in your next post, then went on to create two additional posts instead of editing the first one.

fukawi2 · 2020-09-30 00:23:31

Again, please stop replying to your own thread when no-one else has - there's an edit button for a reason. This is your last warning.

Continuing to explain your symptoms doesn't give us any information to be able to help you. Please read this article: http://www.catb.org/esr/faqs/smart-questions.html

Then come back and give us some useful, detailed information about your system to help us diagnose what's going on.

Arch Linux

#1 2020-09-28 03:53:38

Strange problem on qemu VPS

#2 2020-09-28 04:37:17

Re: Strange problem on qemu VPS

#3 2020-09-28 19:00:16

Re: Strange problem on qemu VPS

#4 2020-09-28 19:39:34

Re: Strange problem on qemu VPS

#5 2020-09-28 19:51:19

Re: Strange problem on qemu VPS

#6 2020-09-28 20:00:52

Re: Strange problem on qemu VPS

#7 2020-09-29 00:46:09

Re: Strange problem on qemu VPS

#8 2020-09-29 14:01:17

Re: Strange problem on qemu VPS

#9 2020-09-29 14:16:20

Re: Strange problem on qemu VPS

#10 2020-09-29 14:27:05

Re: Strange problem on qemu VPS

#11 2020-09-29 14:41:21

Re: Strange problem on qemu VPS

#12 2020-09-30 00:23:31

Re: Strange problem on qemu VPS

Board footer