Constantly monitor the free space on root partition while patching
gnanet opened this issue · comments
Tonight i was patching a Xenserver 6.2, and because the previous patch session was manual, the cleanup was not done. Somewhere in the middle of the 16 patch session the space got to 0 MB free, but the displayed log only showed this:
Downloading: XS62ESP1020.zip
Download Size: 115985387 Bytes > Applying: XS62ESP1020
Uncompressing...
Internal Upload...
Completed: 6057cfe5-811d-46f1-82e1-6da737a0a038Applying Patch 6057cfe5-811d-46f1-82e1-6da737a0a038
Patch failed, code: 1
Command failed: /opt/xensource/bin/xe patch-apply uuid=6057cfe5-811d-46f1-82e1-6da737a0a038 > host-uuid=996b0fb4-c091-47f7-b540-93e99bea0ea4
Failed to apply patch: The patch apply failed. Please see attached output.
output:Secondary validation check...
Patch apply failed for: XS62ESP1020.xsupdate
In order to prevent this, is it possible to predict the space needed to install a patch before "Applying Patch " got started, or better before download starts?
Possibly... Would be easiest if the XML contains file size; otherwise the patch downloaded might be able to establish the patch size and add it to the patch list.
Downloader*
at the time you create the list of patches that have to be applied, you could try to get info about the download sizes.
I am not so deep in python but maybe these can help, both based on urllib:
https://pythonadventures.wordpress.com/2010/10/18/get-url-info/
By default, requests will send 'Accept-Encoding': 'gzip' as part of the request headers, and the server will respond with the compressed content
But, if you manually set the request headers, then you'll get the uncompressed content
But thats only the half of the job, because the space usage consist of these i think:
- extracted patch files size
- patch-backup size
- applied patch size
and predicting is the huge part :(
I was just looking trough my issue list on GH, and it came to my mind that about a month ago i made long postponed updates on a Xenserver 6.X with the latest available version of this script and hit the wall of storage limits. I will collect my notes how i managed to solve the storage shortage, that could maybe used as a base for a guide how to prevent this issue, instead of continously watching the space and to try to stop the patching process before the user is facing a xenserver error message (which as i remember was not descriptive enough to point to the right direction)
Found it, contains a bit of everything that was related to the XenServer 6.2 updates
https://gist.github.com/gnanet/795a6fa2d83d83120e80e65d4dd129d9