Icon pack issues with Synchronization
lishid opened this issue · comments
Describe the bug
We've received a high volume of Obsidian Sync issues related to this plugin generating icon packs with many files, causing Sync to get clogged up.
Ok it seems like this plugin basically download the icon pack as individual files in the plugin folder - I would recommend using a place outside of the vault, or at the very least, pack all the icons together into a single file.
Yes I am aware of this issue. I've highlighted an extra section in the readme file regarding the sync issue. The plugin consists of a setting where you can set the icon folder path. It seems like setting the path to .obsidian/icons fixes the sync issue.
Ah yes, that should fix it as well.
Would it be set like that as default?
Awesome! Then I will write an extra section again in the readme file and close this issue :)
I'm using the Sync and I have the same issue. I followed the instruction from README and set the path to .obsidian/icons, but this still doesn't work for me and Sync logs are empty. It looks like Sync doesn't see those files. Furthermore, I've emailed to Obsidian support, so let's see what they are going to say.
I've got a reply from Obsidian support. They had disabled sync for .obsidian/icons and similar folders on purpose because of this plugin that's creating tons of small svg files which are clogging the Sync. I believe plugin's authors shall consider delivering icons in a tar/zip archive and extract them on demand into the RAM to avoid this issue.
@FlorianWoelki if you'll be so kind to enhance plugin's behavior to not extract all icons to the file system but keep them in zip and extract only necessary on demand, those who are using Sync would appreciate you.
@FlorianWoelki if you'll be so kind to enhance plugin's behavior to not extract all icons to the file system but keep them in zip and extract only necessary on demand, those who are using Sync would appreciate you.
Did it help, if the iconpacks are zipped?
I can not really test it because I am not using Obsidian Sync.
Did it help, if the iconpacks are zipped?
Here is my conversation with Obsidian support:
Me: Will it be helpful to Obsidian Sync if all icons will reside only in 3-5 zip files instead of thousands of small files in a file system which is the case now?
Support: Yes, that should work(I can't comment on the internals of the plugin).
As Licat mentions in this comment packing the icons together should solve this issue.
I can not really test it because I am not using Obsidian Sync.
I gladly can do that. Just let me know.
I also tried another approach to sync my files using open source Syncthing app which is p2p sync solution but it also was struggling to deal with a bunch of small files resulted in 30 min sync of my Obsidian folder which is barely 30 ㎆ of data.
From my experience, putting thousands of tiny (less than ㎅) files is rare a good idea because it leads to a disk usage overhead as well as higher loads to storage's subsystem. For instance:
Four thousand icons with a total size of roughly 3 ㎆ are occupying 17 ㎆ of physical disk space. This is because physically on a disk the smallest allocated block for a single file is 4 ㎅ even if the file itself is 100 bytes only. If you'll put 1 million files 100 bytes each, the file system will allocate 40 ㎇ of a physical disk to store them rather than expected 1 ㎇.
To break this overhead the good practice is always to put tiny files into a zip/tar archive. You even don't need to compress them as you anyway will gain the immediate benefit for the file system i/o and allocation. The simple rule of thumb: whenever it's possible avoid storing tiny files on the file system barely. Some server-grade file systems can cope with that, but common file systems do not.
This still doesn't seem to work for me. I use the .obsidian/icons folder. I manually add one icon pack and I enable background check. Yet, while synchronizing across devices I get the message that missing icons where detetected. I use the same icon pack on all devices. Just one font-awesome-solid.
This still doesn't seem to work for me. I use the .obsidian/icons folder. I manually add one icon pack and I enable background check. Yet, while synchronizing across devices I get the message that missing icons where detetected. I use the same icon pack on all devices. Just one font-awesome-solid.
Could you check if it is just an issue with FontAwesome? Also would be perfect, if you could provide a minimal reproduction vault.
Sure. I've setup a minimal reproduction vault with this icon pack. I have seen this with other icons packs too and that's why I first moved to one. I also used the Tabler Icons and see the same there. I did not add these yet to the minimal repo vault. I sent you an invite on the email connected to your account here on GitHub.
P.s. I seems inconsistent between different vaults. In some cases the same icon used in different vaults is found and in another the same icon is not found.
I'm still seeing syncing issues. Recently reinstalled to give it another go. Icons will work fine on the source machine (i.e. my macbook) but on iOS only about 50% of the icons appear. I have followed all the suggestions in the documentation. https://florianwoelki.github.io/obsidian-iconize/guide/syncing.html
@FlorianWoelki Please consider re-opening this issue. It's not specific to Obsidian sync. I get massive syncing problems with Proton Drive and - to lesser extent - with Nextcloud due to the icon packs. Basically it's clogging up the sync queue and makes it impossible to sync a vault with Proton Drive.
I'm currently waiting for answer from Protons support but for some reason each one of the thousands of *.svg files (each file ~1 kb or even smaller) takes several minutes to hours to sync. I think it's due to encryption but I have no technical knowledge about how any of this works.
I'm pretty sure this issue would be solved if the icon packs would be somehow packed in single archive files as suggested by lishid.