Full backup of VM is failing after 5.100.1
headwhacker opened this issue · comments
If I launch XOA with version 5.100.1, VM backup on an a NFS remote is working fine. This is how the log looks like.
Snapshot
Start: Oct 29, 2022, 03:03:01 AM
End: Oct 29, 2022, 03:03:06 AM
Backups on xxxxx
transfer
Start: Oct 29, 2022, 03:03:06 AM
End: Oct 29, 2022, 04:49:28 AM
Duration: 2 hours
Size: 202.62 GiB
Speed: 32.51 MiB/s
Start: Oct 29, 2022, 03:03:06 AM
End: Oct 29, 2022, 04:49:28 AM
Duration: 2 hours
Start: Oct 29, 2022, 03:03:01 AM
End: Oct 29, 2022, 04:49:40 AM
Duration: 2 hours
Type: full
Launching XOA with any versions after 5.100.1, the same back fails. This is how the log looks like:
Snapshot
Start: Oct 30, 2022, 01:55:33 AM
End: Oct 30, 2022, 01:55:38 AM
Backups on xxxxxx
transfer
Start: Oct 30, 2022, 01:55:38 AM
End: Oct 30, 2022, 03:44:08 AM
Duration: 2 hours
Size: 202.62 GiB
Speed: 31.87 MiB/s
Start: Oct 30, 2022, 01:55:38 AM
End: Oct 30, 2022, 03:44:08 AM
Duration: 2 hours
health check
transfer
Start: Oct 30, 2022, 03:44:31 AM
End: Oct 30, 2022, 05:51:59 AM
Duration: 2 hours
Size: 202.62 GiB
Speed: 27.13 MiB/s
vmstart
Start: Oct 30, 2022, 05:51:59 AM
End: Oct 30, 2022, 06:01:59 AM
Error: waitObjectState: timeout reached before OpaqueRef:17c08a88-d669-4090-b2ba-c7ef45909f0b in expected state
Start: Oct 30, 2022, 03:44:26 AM
End: Oct 30, 2022, 06:02:11 AM
Error: waitObjectState: timeout reached before OpaqueRef:17c08a88-d669-4090-b2ba-c7ef45909f0b in expected state
Start: Oct 30, 2022, 01:55:33 AM
End: Oct 30, 2022, 06:02:11 AM
Duration: 4 hours
Error: waitObjectState: timeout reached before OpaqueRef:17c08a88-d669-4090-b2ba-c7ef45909f0b in expected state
Type: full
Looks like XOA after 5.100.1 is doing an extra steps which is making the backup takes longer causing the failure. Any idea what ha changed in the backup after 5.100.1 that is failing on my setup?
Hi,
The health check should be optional, check your backup job configuration to disable it. What it does is restore selected VMs from backup (as new VM) without network interfaces, wait for the xen tools to report back to host and then destroy restored VM. If your backed up VM has eq. no tools installed or not working properly, the health check part will fail as it'll never report back to host, making the backup job fail as expected.
The heath check is unchecked for the backup. I don't make any changes in the backup config. So it looks like upgrading XOA to > 5.100.1 somehow defaults to heath check enabled?
Hi,
It shouldn't default to health check but seems there's similar reports here: https://xcp-ng.org/forum/topic/6019/health-check/19
Unfortunately it's beyond my ability to fix this inside the container build if there's a bug in backup logic. Clearly it tries to do health check so if you're sure it's turned off everywhere i'd suggest to either make a new backup job to see if it fixes it or rollback to an older version and report your findings to the above community forum thread.