HEIC image files are not scanned for faces if the exif contains faces in the region list
sefininio opened this issue · comments
🐛 Bug Report
📝 Description of issue:
HEIC image files that contain faces in the exif will not be scanned for face recognition.
🔁 How can we reproduce it:
Start scanning a HEIC image file with faces in exif data - it will be added to the library but without face recognition
Please provide additional information:
- 💻 Operating system: unraid
- ⚙ Architecture (x86 or ARM): x86
- 🔢 Librephotos version: latest
- 📸 Librephotos installation method (Docker, Kubernetes, .deb, etc.): docker
- 🐋 If Docker or Kubernets, provide docker-compose image tag: latest
- 📁 How is you picture library mounted (Local file system (Type), NFS, SMB, etc.): local
- ☁ If you are virtualizing librephotos, Virtualization platform (Proxmox, Xen, HyperV, etc.):
I pushed a bugfix, can you check if it now works properly for you?
Sure, dev tag for the entire stack?
Yes :)
@derneuere
OK, so TL;DR: - it works 🔥
Long story - I moved to the dev tag, added some new HEIC photos, and ran a library scan. By the scan ended - the new photos were added to the library but alas - no faces where recognized, nothing in inferred.
I then clicked the blue weight icon on the faces page, that kicked in a new scan and by the end of it - it worked and faces in HEIC files were recognized.
Not sure what is the difference behind the scenes between the library scan and the blue weight icon - but the results were different 🤷
The blue weight icon creates the clusters that represent possible persons. This should also happen at the end of a scan. Without a rebuilding of the clusters, the newly added faces should be in "Unknown - Other". Were you able to see the faces within the photo viewer?
I will close this issue for now and we can create a new one, if we can pinpoint the issue better 😄
@derneuere
Are you using the already existing faces regions from the exif for the recognition? Or are you ignoring them entirely and scan HEICs like JPGs?
Because using the above in dev tag sometimes yields face tags that are... off..., while JPG scans are much more accurate, so I'm thinking maybe the regions in the exif are not positioned correctly and using the same scan mechanism we use for JPGs will produce more accurate results.
For example herem, the face recognized is part of the shirt:
Or here, the neck:
Or here , the coconut:
I am using already existing face regions. In general, the design philosophy is relying on metadata first, then on machine learning / automated processes and lastly on manual workflows.
It looks like the rotation together with the calculation is off. Can you send me the files on discord for debugging? My guess is that these images are flipped, because they are selfies, which I do not handle yet.
Now that you mention it - I re-checked. HEIC images that are not selfies are indeed accurate, while all selfies are off! Nice catch! :)
I'll send some on discord in the relevant thread.
Here is the discord message : https://discord.com/channels/784619049208250388/1188211256319025203/1189907724662747208
After you fix the flipped position for selfies, will it require a rescan and re-tag? or will it also run a migration or something to preserve currently tagged faces and just fix the position?
Also - when do you think this fix and the error person page request fix be part of latest tag?
Should this ticket be reopened? Or open a new one for the selfie?
Not sure yet, if I can find them accurately based on attributes. As this "only" impacts dev, this will probably be a new filter type for selfies in the ui instead of a migration.
I usually do a monthly release, so on sunday or monday this will go into latest.
Would be great if you could open a new ticket, makes it easier for me to write the release notes :)
So, i should hold off scanning for HEIC files until this is merged and the position fix for #1107 is merged, to avoid having to rescan and tag again...