Giters

Turnerj / Quickenshtein

Making the quickest and most memory efficient implementation of Levenshtein Distance with SIMD and Threading support

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Re-look at Threading

Turnerj opened this issue · comments

commented

As long as I can have it allocation-free (or if it must have allocations, make threading optional), there are a few things I could do to make it work.

  1. Use the ThreadPool rather than Parallel.For - assuming that it is faster & allocates less.
  2. With the thread boundaries, have arrays spanning the rows allocated by ArrayPool
  3. Use the presence of a non-zero value in a "previous" boundary to know whether it can start

Having a full array spanning the rows at each boundary prevents any weirdness with having to stall the next thread across manually. All threads (except the first thread) just need to check if the left boundary has completed the row it is up to.

I'd still probably have it on a while (true) loop checking values simply because all locking attempts I've seen are either "too slow" (I don't want to wait milliseconds), allocate and/or require disposal.

I'd probably pass pointer references to both to the strings and their boundaries (so up to 4 pointers). Besides that, each thread would only need to know how many rows (total), how many columns (individual) and their individual start column index.

If I understand everything right, I should be able to use SIMD instructions on top of threading which effectively will allow all my optimizations at once.

commented

With threading, even if it is fast and allocation free, I will want a non-threading path simply because there is no way it is gonna be as fast on short(er) strings. Will need to gauge at what string length it is best appropriate to go from single-threaded to multi-threaded.