PARTUUID Changes after Replacing OS and First Boot
billsidea opened this issue · comments
What is the expected behavior regarding PARTUUID value changes for the related partitions when using the "Replace" feature on the "Maintenance" menu? I had some challenges but was able to keep things working by manually changing "partition" values in the "installed_os.json" file. I'm just wondering if there is an easier way.
I started with a fresh install of PINN (PINN-LITE.img). I am using an Argon One case with a 250GB SATA M.2 V-NAND SSD (Samsung) which is exposed to the Pi as a USB drive.
The initial install included one Raspberry Pi OS and four "Project Space" placeholders. I then did some experimentation with replacing the various Project Spaces as well as replacing installed real OS instances.
In summary, I found that I had to change the PARTUUID references (in "installed_os.json") to "/dev/sdaXX" references. If I didn't do that, the OSs would keep working, but PINN lost track of the partitions (and thus I could not do anything again with those OS instances in PINN like replace them again or use the PINN "Edit Config" feature).
The thing that surprised me most is the changing of the PARTUUID only after the initial boot of the new OS. That basically killed PINN's ability to talk to it, I think.
Am I doing something improperly with the "Replace" feature or is this just a limitation?
I have some step-by-step details below if that helps. I can also provide images of each of these steps if you'd like.
Thanks!
Taking a closer look at one example, these were the steps:
- Replaced an existing Raspberry Pi OS with a new Raspberry Pi OS using the "Replace" feature.
- Before booting the new OS, I confirmed that PARTUUID values were all in sync between "installed_os.json", "/etc/fstab" on the related root partition, and in the PINN Config editor (cmdline.txt "root" value). These values were represented as PARTUUID (e.g. "PARTUUID=3418e696-0c" and "PARTUUID=3418e696-0d").
- Booted to the new OS. It worked fine. I then rebooted and went back into PINN.
- Tried to do an "Edit Config" on the OS in PINN, but it resulted in an "Error mounting boot partition" error.
- Looked at the "/etc/fstab" file on the related root partition and saw that the values had changed to another PARTUUID value (e.g. "PARTUUID=1d47c601-0c" and "PARTUUID=1d47c601-0d").
- Looked at the "installed_os.json" file and saw that those were still the original values (e.g. "PARTUUID=3418e696-0c" and "PARTUUID=3418e696-0d").
- Updated the "installed_os.json" file to specify "/dev/sda12" and "/dev/sda13" instead.
- Rebooted and went back into PINN.
- Tried to do an "Edit Config" on the OS in PINN and it worked successfully. The "cmdline.txt" showed "root=PARTUUID=1d47c601-0d".
- Initiated the "Re-run partition_setup.sh" option.
- Went back into "Edit Config" and saw that "cmdline.txt" had been updated to show "root=/dev/sda13".
Thanks for your detailed report.
This is quite strange as one of the main features of PINN is for it to manage these PARTUUID values for you, and it has been working well for many years. This is the first report of such an issue, so something is wrong somewhere. It sounds like the OS is modifying the PARTUUID after installation. This could be due to a number of things, so I will need to investigate.
You left out some details: which model of raspberry Pi are you using? Which of the 6 variants of the Raspberry Pi OS were you installing (32/64, full/standard/lite)?
It sounds like the OS is modifying the PARTUUID after installation.
When you use e.g. Raspberry Pi Imager to write a Raspberry Pi OS image onto an SD card, the PARTUUIDs end up being those in the image file you downloaded. UUIDs are supposed to be Universally Unique IDentifiers, but they won't be unique if every image has the same PARTUUIDs, and so Raspberry Pi OS generates new unique PARTUUIDs on the first boot.
https://github.com/RPi-Distro/raspberrypi-sys-mods/blob/master/usr/lib/raspberrypi-sys-mods/firstboot#L63
it has been working well for many years
Maybe it's now "broken" because NOOBS is no longer officially supported? 🤷♂️
Could be. I recall there was special noobs detection code at some point.
I used to remove/adapt the firstrun script to prevent the unnecessary root partition expansion, but leaving it didn't seem to make much difference previously.
Maybe the noobs part has been removed now and I need to do some adaptions.
You left out some details: which model of raspberry Pi are you using? Which of the 6 variants of the Raspberry Pi OS were you installing (32/64, full/standard/lite)?
This is on a Raspberry Pi 4 Model B Rev 1.4 with 8GB of RAM. My detailed example was for the "raspios_armhf" (A port of Debian with the Raspberry Pi Desktop) image, but I also experienced this issue with the "twister" (A themed Raspberry Pi OS Desktop merged with gaming v2.1.2) image. The PINN version is v3.8.7 (Built: May 6 2023).
There is another point worth mentioning about when you're in this situation where you've replaced an OS using PINN's "Replace" feature but have not yet worked around the issue (by manually updating the "installed_os.json" file). The OS will keep working and you can still boot into it. However, if you try to do another "Replace" on that instance, you will get this type of error: "Cannot Reinstall/Replace raspios_armhf. Partition not big enough for new image." I believe it is essentially the same technical issue as I described above, but the message might take you in another direction for troubleshooting. I think I ran across some other issues/posts where people were seeing this message. I just wanted to point that out in case it may help with any other issues you may be fielding.
Yes, that's certainly relevant, thanks.
Does "re-running the partition_setup.sh script" from the fix up menu sort this out, without you having to mess with installed_os.json?
@lurch - The firstboot script still checks for the presence of NOOBS, but doesn't take this into account before calling fix_partuuid. Looks like this has been an issue since 20220901 release for rpi_imager customisations.
I have raised an issue for XECDesign to consider.
Does "re-running the partition_setup.sh script" from the fix up menu sort this out, without you having to mess with installed_os.json?
Unfortunately, no. In this scenario, running that script results in the following:
Checking for partition_setup.sh scripts...
Executing /settings/os/raspios_armhf/partition_setup.sh
part1= id1=LABEL=RECOVERY
SETTINGS
boot
root
boot64
root64
boot640
root640
boot1
root0
boot0
twister part2= id2=LABEL=RECOVERY
SETTINGS
boot
root
boot64
root64
boot640
root640
boot1
root0
boot0
twister
raspios_armhf: Error executing partition setup script
raspios_armhf partions_setup.sh script error
DONE - some setup script errors detected
FYI, I did another test of the same scenario, but I did the workaround slightly differently. Instead of updating "installed_os.json" with the "/dev/sdaXX" values, I updated "installed_os.json" with the newly-regenerated PARTUUID values (as taken from "/etc/fstab"). This also appears to fix the issue. After doing that, I can once again use the PINN features for that OS (e.g. "Edit Config"). If I then do the "Re-run partition_setup.sh" from the fix menu, it works and updates "cmdline.txt" and "/etc/fstab" with the "/dev/sdaXX" values. However, the "installed_os.json" file continues to have the (correct) PARTUUID values and everything works fine.
Just as an extra bit of info as to why these PARTUUID values are being used instead of always using /dev/sdaXX values... (in case you're not already aware of this)
If you only have one USB drive plugged in (i.e. your SSD) then it'll always appear as /dev/sda. But if you plugged in another USB drive (e.g. a USB flash drive) and then rebooted, and if that flash drive initialised faster than your SSD, then your USB flash drive would be /dev/sda but your SSD would now be /dev/sdb and any references to a /dev/sdaXX partition wouldn't work 😉
In the early days of NOOBS (which PINN has evolved from) this wasn't a problem, because the Raspberry Pi was only able to boot from SD card, and that always appears as /dev/mmcblk0 regardless of what you've got plugged into your USB ports. Booting directly from USB was IIRC a feature added by the Pi 3.
Thanks for that info, @lurch. That makes sense. Although convenient to reference the /dev/sdaXX values because they are more stable than the PARTUUID values in one sense, they are not as stable in a "multi-USB-drive" environment. My testing does show that referencing the "PARTUUID" values does work (once manually fixed in "installed_os.json" after first boot), but I'm just not sure how PINN would be able to sense the PARTUUID change upon first boot of the OS. Maybe theres a way to alter that "Re-run partition_setup.sh" feature to somehow try to parse that OS's "/etc/fstab" file (and then use those PARTUUID values rather than the /dev/sdaXX values going forward).
We just need to stop Raspios (and others based on it) from modifying the diskid when in a multi-boot environment. I'm working on it.....
somehow try to parse that OS's "/etc/fstab" file
No need to mount the rootfs partition and try parsing the enclosed /etc/fstab, you can just read the PARTUUID directly from the partitions (it's part of the metadata / header-info).
We just need to stop Raspios (and others based on it) from modifying the diskid when in a multi-boot environment.
Out of curiosity, if those changes were made, if a PINN user installed multiple copies of the same OS image on the same drive, would there be a PARTUUID clash that would potentially break things? I'm willing to help test that use case if you think it's valid and have any concerns about it.
Out of curiosity, if those changes were made, if a PINN user installed multiple copies of the same OS image on the same drive, would there be a PARTUUID clash that would potentially break things?
No because the PARTUUID is DISKID-PARTNO, which will be unique.
I also have the same problem and have been annoyed about it for days. I am using the beta on a Raspberry Pi 5.
However, I have the impression that this PARTUUID only changes after I have booted ALL installed operating systems at least once.
In my opinion, the use of the PARTUUID is therefore complete nonsense and should finally be changed as quickly as possible instead of constantly adding gimmicks that hardly anyone needs anyway.
I'm sorry, but user-friendliness is something else!
I wonder why the PARTUUID changes anyway, as it is derived from the disk identifier. Change it to UUID and the problem would be solved!
@mkreisl - Which OS are you installing (exact name, version, release date would help)
I don't think switching to UUID will totally help. The problem was that everyone who downloads the standard image would have the same PARTUUIDs or UUIDs. So to make these unique, a startup script in the OS is run to alter the PARTUUID during first boot. This is fine when it is the sole OS on the drive, but under a multi-boot scenario where PINN records the PARTUUIDs to know which partition to boot, changing the PARTUUID after the fact is going to cause problems. Especially since changing the diskid affects all the PARTUUIDS, so all of theOSes will be affected. Changing to UUID and adjusting it on first boot would just cause that OS to be affected, thus lessening the impact, but it would still cause a problem for that OS.
For Raspios, this has already been fixed since it no longer changes the PARTUUID if it detects it is installed under PINN, since PINN will have already created a unique diskid for the drive, so all PARTUUIDs will be unique by design.
If you are having problems with another OS, then any script needs to be modified to not change diskids if it detects NOOBS/PINN, or the partition_setup.sh script can be modified to remove or adapt such a script.
@mkreisl - Which OS are you installing (exact name, version, release date would help)
I had tried several times, I will now describe my last attempt here:
The hardware I used is a Raspberry Pi 5/8GB with external 250GB USB3 disk, no sd card.
I had installed XBian in the 32-bit and 64-bit version, as well as ubuntu2310 and raspios_arm64
So that I could install my XBian versions at all, I had changed the cmdline.txt as follows:
runinstaller quiet ramdisk_size=65536 root=/dev/ram0 init=/init vt.cur_default=1 elevator=deadline repo_list=http://sourceforge.net/projects/xbian/files/noobs/repo_list.json ssh loglevel=2 sdhci.debug_quirks2=4
I have also activated ssh, otherwise everything corresponds to the original pinn-lite.zip from the beta folder
After all 4 operating systems finally landed successfully on the disk, I started Pi-OS first and set it up briefly. after this was done, I booted it again sucessfully.
Then I started the 32-bit XBian and then the 64-bit XBian. I proceeded in the same way as with Pi-OS, set up briefly and booted again.
In between I booted Pi-OS again and again to check whether the PARTUUID's had changed, they hadn't so far
The last thing to go was Ubuntu. After I had set everything up so far, I carried out the update offered there without rebooting and then rebooted immediately.
But it didn't want to boot anymore, it ended up in the initramfs command line, ext4 couldn't be mounted anymore because something went wrong with the fs.
In any case it could be repaired with e2fsck. but I had done that on another pc.
Anyway, the disk identifier had now changed and thus the PARTUUIDs, so the end of booting for the systems that use the PARTUUIDs
I don't think switching to UUID will totally help. The problem was that everyone who downloads the standard image would have the same PARTUUIDs or UUIDs.
I really don't understand what you mean. It's about the newly installed systems by PINN, for this purpose 2 partitions are created for each operating system and thus these two partitions receive a unique UUID, that's the point of the UUID, that it is really unique.
I also don't know how to change the disk identifiers and thus the PARTUUIDs by normal means, but obviously this happens as you have described.
I have added an extension to partition_setup.sh that replaces the PARTUUID's passed with the UUID.
convert_partuuid() {
uuid=''
if echo "$1" | grep -q 'PARTUUID'; then
p=$(echo $1 | awk -F = '{print $2}')
[ -n "$p" ] && uuid=$(blkid | grep "$p" | awk -F UUID= '{print $2}' | tr -d \" | awk '{print $1}')
fi
[ -n "$uuid" ] && echo "UUID=$uuid" || echo "$1"
}
part1=$(convert_partuuid $part1)
part2=$(convert_partuuid $part2)
Then whoever wants to can change the disk identifier and thus the PARTUUIDs, the installed system then cares very little about this. Since then there has been no problem.
But of course you can't do that with other people's things
Just to be clear:
- There is no reason for the PARTUUIDs to change, and they shouldn't.
- PINN is not changing the PARTUUIDs
- Most likely, this is due to the initial scripts in Raspios., although as far as I am aware, that bug has been fixed. See https://github.com/RPi-Distro/raspberrypi-sys-mods/blob/3af395f60cfc5061d72ea79f3d33bf0ccdac29a8/usr/lib/raspberrypi-sys-mods/firstboot#L25, which is why I was asking for the specific version of the OS.
Yes you are right, UUIDS (and PARTUUIDs) should be unique. The problem is that an OS is normally distributed as an IMAGE file, which contains the UUIDS/PARTUUIDS/DISKIDs. So every installation is essentially a clone. Normally this is ok until you put 2 or more of those cloned partitions on the same system. There were some instances of this, so RPT chose to change the diskid during the initial configuration of the OS. This is fine on a single OS drive, but catastrophic in a multi-boot environment because it changes the PARTUUID of all the other OSes.
So, you can see in the link above that they now check if the boot partition is not 1 (in which case it is running under NOOBS/PINN) and then they do not fixup the partuuid. So it should not happen anymore. As it is still happening to you we need to debug that.
I don't think it is related to Ubuntu because it references its partitions via labels (not ideal for other reasons) and so would have no need to change the disk id.
I don't think it's just due to a script running amok in Pi-OS, because I made my first attempts (it must have been at least 4 attempts or more) all without Pi-OS, so only XBian and Ubuntu.
It also occurred there. I also suspected XBian, because when the root partition is at the end, it shrinks the root partition and appends a swap partition. but I checked the script and couldn't see that this changes the disk identifier. As I said, I don't know how this is supposed to work. that would make the whole thing absurd anyway.
The use of fdisk to change this has already been investigated, but I don't have an fdisk that allows you to change this id using the i command
I don't think it is related to Ubuntu because it references its partitions via labels (not ideal for other reasons) and so would have no need to change the disk id.
I agree with you
I will try to follow your installation steps to see if I can replicate your issue. It may take a little while....
Re-reading your procedure, maybe it was the ubuntu update that killed it? Was it version 23.10?
Re-reading your procedure, maybe it was the ubuntu update that killed it? Was it version 23.10?
Yes
I have now carried out the procedure again, but after starting Pi-OS 64-bit and its configuration, update with subsequent reboot, during which I checked the PARTUUID, I immediately started XBian 64-bit, which, since it was installed at the end of the disk, created a swap partition.
I then checked the PARTUUID without rebooting and it had changed.
So now again, the question is, how can I change the PARTUUID?
Tomorrow I will carry out a few more tests, but I stick to my opinion: the use of PARTUUID's is in my opinion absolute nonsense
Has Xbian always created such a swap partition? That's not really compatible with the way PINN works, although it shouldn't make a difference if it only happens if it is at the end of the disk. It might be better to explicitly allocate a swap partition for it no matter where it is located.
So now again, the question is, how can I change the PARTUUID?
You change the PARTUUID by changing the diskid using fdisk or similar.
PARTUUID = diskid-partitionNumber
Yes, this has always been the case. But only if the XBian root partition is at the end of the disk, otherwise it can't work.
Tomorrow I will suppress creation of the swap partition, this can be done simply by deleting a parameter in cmdline.txt. Let's see what happens then.
You change the PARTUUID by changing the diskid using fdisk or similar.
PARTUUID = diskid-partitionNumber
As I said, I have no idea how this should be done.
By the way, parted is used, which cannot do such a thing AFAIK
It might be better to explicitly allocate a swap partition for it no matter where it is located.
Can this be done by PINN? XBian only tries to create a swap partition if there is none. That would of course be an elegant solution
The problem seems to be caused when Xbian creates its swap partition.
I see Xbian is on Github. Can you provide a link to where it creates the swap partition?
I think I found it at https://github.com/xbianonpi/xbian-package-initramfs/blob/master/content/etc/xbian-initramfs/cnvres-code.sh
So my guess is that parted (or mkswap) updates the diskid when it creates the swap partition, but that would need to be confirmed.
You can get PINN to create a swap partition by adding it to partitions.json, then there will be no need to create one, and it would would be created even if Xbian wasn't the last partition.
You have found the right place.
I have now prevented the creation of the swap partition and thus the disk identifier no longer changes
Now I just have to find out how to define a swap partition via PINN. shouldn't really be rocket science 😄
Now I just have to find out how to define a swap partition via PINN. shouldn't really be rocket science 😄
Unfortunately obviously already
First the good news:
Somehow I managed to get PINN to create a swap partition which is then recognized and used as such by XBian.
Now the bad news:
The installation does not run without an error message.
- if I don't define a tar file, PINN complains that the swap.tar is missing
- if I define a dummy tar, as an interim solution I used the link from the data partition, it complains that it can't mount the partition
The entry "emptyfs": true does nothing either
In both cases there is an error message, but the installation continues invisibly but in the background. at some point the message appears that the installation was completed with errors
I have also noticed that although I have set the Utility group for the Swap, Swap always appears in the General group.
This also happens with Data Partition
This is the latest version:
os_xbian.json:
{
"Group": "Utility",
"description": "Adds an 512MB swap partition to the partition layout.",
"icon": "http://sourceforge.net/projects/xbian/files/noobs/swap/swap.png",
"nominal_size": 512,
"os_info": "http://sourceforge.net/projects/xbian/files/noobs/swap/os.json",
"os_name": "Swap Partition",
"partitions_info": "http://sourceforge.net/projects/xbian/files/noobs/swap/partitions.json",
"supports_backup": false,
"emptyfs": true,
"tarballs": [
"http://downloads.raspberrypi.org/data_partition/data.tar.xz"
]
}
os.json:
{
"name": "Swap Partition",
"description": "Adds an 512MB swap partition to the partition layout.",
"slice_exclude": true,
"bootable": false
}
partitions.json:
{
"partitions": [
{
"label": "swap",
"filesystem_type": "swap",
"partition_size_nominal": 512,
"want_maximised": false,
"mkfs_options": "",
"uncompressed_tarball_size": 1
}
]
}
All in all a totally frustrating affair 😢
Cant remember how to manage swap files. I'll check later.
You should just need to add an extra partition definition to your partitions.json file. Here is an example for your arm64 version:
{
"partitions": [
{
"label": "xbianboot",
"filesystem_type": "FAT",
"partition_size_nominal": 100,
"tarball" : "https://sourceforge.net/projects/xbian/files/noobs/xbianboot-arm64_rpi5.tar.xz",
"want_maximised": false,
"uncompressed_tarball_size": 50
},
{
"label": "xbianroot",
"filesystem_type": "raw",
"partition_size_nominal": 3500,
"tarball" : "https://sourceforge.net/projects/xbian/files/noobs/xbianroot-arm64_rpi5.xz",
"want_maximised": true,
"mkfs_options": "",
"uncompressed_tarball_size": 2800
},
{
"filesystem_type": "swap",
"partition_size_nominal": 512,
"want_maximised": false,
"empty_fs": true
}
]
}
Remember to add the partition size to your os/os_list.json file nominal_size parameter.
(I've not tried this yet, so it might need some adjustment)
Thanks for you help
This may work, but I don't even try it, because it may well happen that someone wants to install both the 32-bit and the 64-bit version and he would then probably have 2 swap partitions on the disk.
So I find a separate swap configuration more sensible, especially if you have enough memory, a swap partition is not really necessary.
I built a workaround for this today that works as far as I can see. It took me hours to get it working, you have to be pretty tricky.
Now I have to take a break from it again, otherwise I'll go crazy 😄
OK.
JFYI You had a typo - replace emptyfs with empty_fs
Also, if you set filesystem=swap, then PINN will format it for you with mkswap.
JFYI You had a typo - replace
emptyfswithempty_fs
WTF, I took that parameter from your wiki and there is emptyfs
Also, if you set filesystem=swap, then PINN will format it for you with mkswap.
Yes, I have already found that out. however, it never worked without an error message from PINN.
Oops, Sorry! I'll have to update the wiki! Trouble is, the variable name I use is called "emptyfs" and I'm easily confused 😉
I'll try and test the swap file out later.
EDIT: wiki updated.
Yes, I have already found that out. however, it never worked without an error message from PINN.
It seems no-one has used the swap feature before. Just as well as it needs some work to remove it from normal post installation processing (PINN tries to mount it and write an os_config.json file to it like all other boot partitions 😮 ).
At some point it's always the first time 😃
I have updated PINN's usage of mkswap, so it should work now. PINN beta should recognise the update being available and update to 3.8.9d.
I have created a "swap" OS which I will make available soon.
My swap OS is now available. Please check it with the latest version of the PINN Beta 3.8.9d.
I think this is a slight improvement on your "swap partition" as it is a proper swap partition.
If you're happy with it, please remove your swap_partition from your repo_list to avoid confusion and I will add your repo_list to PINN.
Thanks.
I think this is a slight improvement on your "swap partition" as it is a proper swap partition.
If you're happy with it, please remove your swap_partition from your repo_list to avoid confusion and I will add your repo_list to PINN.
Works great. And I removed my swap configuration from the list
Thanks for that. This is undoubtedly the better solution
I'm afraid I have to correct myself 😢
The swap partition is created and mounted correctly. That's ist ok, but everything else no longer works.
The fstab(s) and probably also the cmdline.txt of the installed systems are no longer changed . I have checked this for 32-bit XBian, 64-bit XBian and PiOS 64-bit lite. In PiOS, for example, the PARTUUID entries are not correct, which for me is an indication that they have not been changed
This means that the systems do not boot correctly.
So now everything is worse
Although my "swap" os will create a swap partition, I'm not sure how the various OSes will link up to it.
Will XBian automatically find the swap partition, or would you have to add it explicitly to the fstab? In which case you may have to discover and add it in a startup script as the swap partition may not have been created when the XBian partition_setup.sh runs.
BTW - In doing the change for mkswap, I have inadvertently removed some ciphers for SSH, so I need to revert the kernel8. I will check I haven't screwed up the partition_setup.sh execution mechanism at the same time, but it won't be until tonight unfortunately.
XBian is intelligent in this respect and automatically includes all existing swap partitions that it finds on the disk. But I think I said that before.
I checked further, the fstab of PiOS is definitely not adjusted, its file date is still from last year instead of being from today
I have reverted pinn-lite.zip for now, so partition_setup will work but mkswap won't. I'll look into it tonight.
@mkreisl - please could you try https://sourceforge.net/projects/pinn/files/beta/pinn-389e.zip/download. Hopefully I have fixed the recent issues
From my point of view, I can say that everything is working again as before