shotit / shotit-worker

Four core workers of shotit: watcher, hasher, loader and searcher.

Home Page:https://shotit.github.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Low CPU usage when `shotit-worker-loader` loads data to the vector database

Leslie-Wong-H opened this issue · comments

This JSON polishment operation is just single-threaded and slow.

image

Proposal:

Need to make use of js workers to speed up the index loading process.

Inspiration:

Promise.all to parallel

PR #45 fixed