fsck.vfat needed
bill-mcgonigle opened this issue · comments
Over on #62 folks are seeing no video, they run a disk repair on the sdcard, and then the camera works fine.
By total coincidence the UPS on my camera went up in smoke this morning, the openmiko up on my roof rebooted uncleanly, and I had no video feed.
/sdcard was mounted read-only due to errors at boot time. e.g.
/root/restart-rtspserver: line 1: can't create /sdcard/debug/rtsp-2022-06-09-12-50-25.log: Read-only file system
I went up on the roof with my screwdriver and a USB-C sdcard reader, got it mounted on my Android phone, and safely ejected it. That seems to have fsck'ed the sdcard and now it boots just fine.
Copying from my comment there:
it doesn't make sense to not be able to repair a filesystem on these types of devices (they mostly run in unreliable power situations). It looks like for large SD cards on embedded a low-memory branch of dosfstools is required: https://github.com/arnout/dosfstools/tree/reduce-memory-v2 . That builds with just a couple warnings on my Debian amd64 machine. I don't have cross-compiling currently set up. Does anybody here have the ability to cross-compile a static binary?
It makes sense to fsck the filesystems on boot, especially since fat32 is so fragile. Getting up on the roof would be tricky mid-winter!
Here is a binary I compiled. Just rename it to fsck.vfat and chmod +x it.
It's hacky for now until I get it installed as part of base image.
Is this the reduce-memory branch? FYI, I got:
# fsck.vfat -n /dev/mmcblk0p1
CP437: Invalid argument
fsck.fat 3.0.28 (2015-05-16)
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
Automatically removing dirty bit.
malloc:Cannot allocate memory
Certainly at boot time there may be more memory available.
Is this the reduce-memory branch?
Not sure. Is there a branch of fsck.vfat that has lower memory requirements?
This one seems to be the most popular for low-memory devices:
https://github.com/arnout/dosfstools/tree/reduce-memory-v2
Not sure if you used the link in my report or the one in the standard tree. Nokia also has a patched version if fsck.vfat that takes a different approach (for their phones, I think). The stock loads the whole disk data structure into RAM IIRC, and the low-mem version doesn't.
I see /sdcard can be other filesystems in the next alpha. I'll totally switch to ext4 if there's a ulibc mips e2fsck!
I was getting a ton of these in dmesg
[1151418.481147] FAT-fs (mmcblk0p1): error, clusters badly computed (3203 != 3202)
[1152743.599067] FAT-fs (mmcblk0p1): error, clusters badly computed (3204 != 3203)
[1153829.870019] FAT-fs (mmcblk0p1): error, clusters badly computed (3205 != 3204)
[1155040.555268] FAT-fs (mmcblk0p1): error, clusters badly computed (3206 != 3205)
So I loaded fsck.vfat onto my openmiko and ran it, and it seems to have fixed it.
Anyone have the reduce-memory-v2 binary compiled for this that I can snag?
Awesome, @martinbutt - were you able to get the reduce-memory-v2 build of mkfs.vfat?
@bill-mcgonigle, no. The complied binary was coming out much larger than than the upstream master branch. That and the fact that branch is 8 years (314 commits) behind the upstream master led me to go with the upstream one. I have't had any issues with it.
@bill-mcgonigle, no. The complied binary was coming out much larger than than the upstream master branch. That and the fact that branch is 8 years (314 commits) behind the upstream master led me to go with the upstream one. I have't had any issues with it.
How large of a microSD card have you tried with it? Pretty sure it just segfaults on larger ones due to the memory constraints.
I've been using 32Gb cards. I have a 512Gb I can test with.
I was able to reproduce the malloc:Cannot allocate memory message with my 512Gb card. I complied the reduce-memory-v2 branch and got the same message. The binary is here for reference: fsck.fat.txt
I tried with the very latest version an I get Filesystem is too large. Here's that binary: fsck.fat.txt
If you find a fix for this, I'm happy to try to implement it.
I decided to merge together all of my changes from the last six months and release them for testing. This includes my fixes for adding fsck. Testing is appreciated, and you can file an issue in that repo if you have any problems.
Spent a little more time on my hard-to-reach camera. I put a wifi power switch on it and it gets monitored and reset if it stops responding to pings for 5+ minutes, but vfat gets mad about that. Using this fsck.vfat:
md5sum sbin/fsck.vfat
7e6a1319edc16c0eba6811bf3a441ba7 sbin/fsck.vfat
with a 16GB Sandisk. I'm running this script:
# cat usr/sbin/fsck-sdcard-if-ro
#!/bin/ash
RO=$(/bin/grep /sdcard /proc/mounts | /usr/bin/uniq | /bin/grep -q 'ro,')
if [ $? -eq 0 ] ; then
SYS=$(/bin/grep /sdcard /proc/mounts | /usr/bin/uniq | awk '{print $3}')
DEV=$(/bin/grep /sdcard /proc/mounts | /usr/bin/uniq | awk '{print $1}')
logger -s -t sdcard -p local0.emerg "/sdcard is mounted read-only."
if [ -x /sbin/fsck.$SYS ]; then
logger -s -t sdcard -p local0.emerg "attempting repair of /sdcard."
/sbin/fsck.$SYS -a $DEV
fi
logger -s -t sdcard -p local0.emerg "rebooting to try to clear read-only /sdcard."
/sbin/reboot
fi
every minute from cron. Very hackish but it only takes 0.01s of wall clock time to check so not the worst. So far it's repaired vfat successfully a few times on reboots.