Sync/Async APIs
masylum opened this issue · comments
Problem Description
According to my profiling, most of the time spent on ingesting documents is spent on "run microtask". Particularly the insertScalarBuilder
is specially hot, and it has a lot of faux async/await that is actually doing sync work by default. I understand this is because you want to have an async API for people to provide their storage, but it makes the default in-memory much slower.
Proposed Solution
Perhaps an option would be to have 2 APIs, a sync and an async one? This is yet another example of https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function, which sucks, but I don't make the rules.
Alternatives
No response