Local embeddings transformers Error: no available backend found (stuck on 'Loading Smart Connections...')
alk0 opened this issue · comments
2.1.67. Stuck on "Loading Smart Connections..." in the plugin window, "Refresh notes" does indeed show notifications about scanning (and finishing the scan) for new notes, but it doesn't change anything. "force_refresh" - haven't tried yet, since it takes hours on this PC. I did it before, and it finished the job, and everything worked after that (yesterday). Any ideas where to look for any debug info or something?
UPD: The console shows a ton of errors, but this (repeating) one may be of interest:
cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2/dist/ort-wasm.wasm:1
Failed to load resource: net::ERR_QUIC_PROTOCOL_ERROR"
Can it be the reason?
Hi @alk0
It's possible, you can test this in one of two ways:
-
switching to OpenAI models for embedding in the settings
-
using the https://github.com/brianpetro/smart-connect software to offload the embedding process (this will prevent loading transformers in Obsidian).
For 2, you will need to restart Obsidian after opening the Smart Connect app for it to take effect.
Thanks for your help in figuring this out
🌴
- is unavailable for me (geo-restricted)
- last time I checked, it didn't work for me, it requires a library I don't have on my system (onnxruntime, I believe; Manjaro Linux, Arch-based, run as an AppImage)
... and about cdn.jsdelivr.net - seems there is indeed a problem on my or their side (or somewhere in between), I can't even download ort-wasm.wasm, it starts, then drops to dial-up modem speeds, then stalls. So... I guess, not exactly obsidian-smart-connections' problem.
This might help with Smart Connect on Linux
sudo ln -s /home/ubuntu/dev/smart-connect/app/node_modules/onnxruntime-node/bin/napi-v3/linux/x64/libonnxruntime.so.1.14.0 /usr/lib/libonnxruntime.so.1.14.0
About that smart-connect error
sc_env.js:282 Object
node:electron/js2c/renderer_init:2 Uncaught (in promise) Error: Error invoking remote method 'smart-embed-batch': Error: libonnxruntime.so.1.14.0: cannot open shared object file: No such file or directory
at a.invoke (node:electron/js2c/renderer_init:2:7022)
at async handle_embed (sc_env.js:284:16)
sudo ln -s /home/ubuntu/dev/smart-connect/app/node_modules/onnxruntime-node/bin/napi-v3/linux/x64/libonnxruntime.so.1.14.0 /usr/lib/libonnxruntime.so.1.14.0
Ugh, well, I had to 1) unpack AppImage, 2) install some node.js sh*t for the next unpacker, 3) unpack app.asar, 4) find that damn lib and link it to /usr/lib - with all that, smart-connect does indeed work (it decided to redo the embeddings once again - despite them being by the same model - and it didn't resume when interrupted... but at least it's something)
Okay, it's still unusable - now with smart-connect it redoes embeddings from scratch each time Obsidian is restarted.
Maybe it has something to do with the fact that I have Block Embedding Model set to None in the settings? And the console is full of these:
plugin:smart-connections:2421 error parsing blocks for note: some_note_file_name.md
plugin:smart-connections:2422 TypeError: Cannot read properties of undefined (reading 'dims')
at SmartBlock.update_data (plugin:smart-connections:2460:108)
at SmartBlocks.create_or_update (plugin:smart-connections:390:30)
at eval (plugin:smart-connections:2417:31)
at Array.forEach (<anonymous>)
at SmartBlocks.import (plugin:smart-connections:2416:18)
That's possible. Maybe try bge-micro for blocks
Some more testing... I don't know what is wrong, but - embeddings seem to be never saved (and never read?). (A message in console said it "saved" them, but no index files were changed or created where I expected to see them.) I tried another vault, with 2.1.58, updated it to 2.1.67, it immediately forgot about existing embeddings, created new, but they existed only until next time plugin was initialized (either by restarting Obsidian, or by trying to reconnect to SC). I rolled back to 2.1.58, and embeddings are being saved again, including on pausing (to .smart-connections/smart_notes-bge-micro-v2.ajson in my case). I absolutely can't afford recreating them every time I open Obsidian. So... I don't have many options. Stop using the plugin, or remain on 2.1.58, or install 2.1.66, until it's resolved somehow.
@alk0 The saving issue was fixed in v2.1.68 and #634 (comment) was fixed in v2.1.69.
After updating, you will need to re-embed again, but after that, everything should work as expected.
🌴
@alk0 The saving issue was fixed in
v2.1.68and #634 (comment) was fixed inv2.1.69.After updating, you will need to re-embed again, but after that, everything should work as expected.
🌴
First-time user who just downloaded v2.1.69 - I'm still stuck on "Loading Smart Connections...".
I'm using Nomic-embed-text-v1.5 as both the Notes and Blocks embedding model, with max tokens set to 8192. The initial process seems to have finished - the .smart-connections folder is full of .ajson files. Tried force_refresh. Have RAM and CPU.
However - I am using an old Windows 7-compatible version of Electron, along with the latest Obsidian's resource files. Might try with latest Obsidian installer on Windows 10.
Also, where are the local models stored? I tried searching for files with "nomic" in the filename, and nothing came up... are my models and notes being processed remotely? That is the LAST thing I'd want.
Also, where are the local models stored?
Somewhere in Obsidian's Electron caches, in my case under .config/obsidian/Service Worker/CacheStorage, with "random" filenames.
After updating, you will need to re-embed again, but after that, everything should work as expected.
Confirmed, works "almost" as expected with Smart Connect. Haven't tested with the local transformers yet, since cdn.jsdelivr.net is still misbehaving for me :(
One thing though - on <=2.1.66 embeddings were saved separately for each model, so you could have two (or more) sets of them, updating only new ones separately when switching models. Now, if I want to try a different model, it means that I can't return to the previous one without redoing all embeddings again. (Unless I find and save them manually, which I haven't tried yet.) It's not a bug, but... something to be aware of, for a user. (Needs more testing; DISPROVEN - embeddings are saved separately and don't require re-embedding)
@iFUCKINGHATEcomputers as a new user, I highly recommend getting things working with the default BGE-micro-v2 local model before trying others.
And no, processing is not happening remotely just because you can't find the model file. This process is handled by transformers.js, a trusted package produced by HuggingFace for implementing local models. I recommend further inquiring there if you need to know specifics about where the model is stored (likely in a temp file within the browser data, a location that often changes depending on OS, which is likely why I couldn't find the exact location with a quick search).
Also, for the sake of this being a friendly community centered around computers, maybe consider changing your username to something respectable 😊🌴
@alk0 do you think it's safe to say this is a linux-specific issue? If so, I'd like to append "(linux)" to the end of the issue title, i.e. Stuck on "Loading Smart Connections..." (linux)
@brianpetro Nevermind, it seems I had to wait a long while (1 hour?) for the plugin to fully prepare - now it's asking me to start embedding. Also, Ted Kaczynski was right.
@iFUCKINGHATEcomputers This is a place for the Smart Connections community to communicate issues regarding the Obsidian plugin. This should be done respectfully. And if you can't be polite, respectful, or you insist on promoting your politics, then you are not welcome here 🌴
@alk0 do you think it's safe to say this is a linux-specific issue? If so, I'd like to append "(linux)" to the end of the issue title, i.e.
Stuck on "Loading Smart Connections..." (linux)
Not sure yet if it's linux-specific. It may be also regional-, or provider-, or even Electron-host-specific, though (inability to load this particular resource from JSDeliver). Can you have a look at the attached log, please? Maybe you'll see more than I can? I have zero experience in JS programming. From the user perspective it still looks as "stuck on loading Smart Connections", when Smart Connect is not running. When it is running, everything seems to work now (after monkey-patching /usr/lib - this part is Linux-specific, but it should go to Smart Connect github issues, I guess...)
obsidian.md-1717688827375_redacted.log
@alk0 thanks for the logs.
I see the issue, but it seems highly dependent on specific circumstances, and may not be anything either Smart Connections or Smart Connect can address. There are similar issues in the transformers.js repo, see https://github.com/search?q=repo%3Axenova%2Ftransformers.js+Error%3A+no+available+backend+found&type=issues
Maybe we can update the title to something like "Local embeddings transformers Error: no available backend found"
And if you see anything in those issues that I might be able to help with, let me know.
Let's take a break for now, shall we? I'll report if I find anything worth looking into. And I don't mind changing the title, once the issue is narrowed down, supposedly.
@alk0 I updated the title because "Stuck on 'Loading Smart Connections'" can apply to many many processes that happen before loading. Please revisit whenever you have a chance. Thanks for your help in solving this 🌴
@brianpetro Regarding that:
cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2/dist/ort-wasm.wasm:1
Failed to load resource: net::ERR_QUIC_PROTOCOL_ERROR"
Seems the problem with JSDelivery has resolved for me here (somehow) after a couple of days, so now I don't have this particular problem anymore, local transformers work as expected. I'll leave it up to you whether to close the issue. If anything, it can be re-opened.