[questions] benchmarks; scores VS penalties

[Hannes Hauswedell]

Hi,

this looks like a great project! I have two questions:

1. Do you have benchmarks against libraries that implement NW? Like parasail and seqan? You say that auto-vectorisation of the library works well, do you have benchmarks for this? Do you assume that this is comparable to the speed-up that libraries with explicit vectorisation (inter-sequence batch vectorisation or anti-diagonals) get?
2. You are currently only offering penalties (negative scores) and the match-"score" is fixed to 0. Is this a limitation of the algorithm, or would non-negative scores work as well? I think that would be a requirement to get Protein scoring matrixes to work?

Thanks!

[Santiago Marco-Sola]

Hi,

Thanks for the interest in the tool! Let me go through your questions and don't hesitate to share your comments and further questions.

Do you have benchmarks against libraries that implement NW? Like parasail and seqan?

Yes. Probably, a good summary can be found in the WFA publication (Table 2).

Nevertheless, you can always run your own tests and compare. The benchmark branch incorporates SeqAn into the align-benchmark tool, together with other alignment methods, including all WFA variations.

You say that auto-vectorisation of the library works well, do you have benchmarks for this?

Yes, I believe so. The ASM code generated by the compiler is the code I would expect to have if I had to program it myself using intrinsics. We have explored the ASM generated on x68-64+AVX and ARM+NEON/SVE. In the past, we confirm the performance gain by doing some benchmarks on this (i.e., scalar vs SIMD). Nevertheless, a very skilled programmer might be able to gain some performance by doing it by hand (against the compiler); you never know.

Do you assume that this is comparable to the speed-up that libraries with explicit vectorisation (inter-sequence batch vectorisation or anti-diagonals) get?

No, I would not make such an assumption. First of all, SWG/NW algorithms have a different computational pattern and workload than the WFA algorithm. Roughly speaking, the WFA computes wavefront vectors of increasing length up to the optimal score (i.e., the number of WF vectors computed depends on the distance/score between the sequences).

Considering this, for the WFA:

• Inter-sequence parallelization: Yes, straightforward.
• Inter-sequence vectorization: Not straightforward as each pair of sequences has a different optimal score; thus, different computation work (not regular).
• Intra-sequence parallelization: Yes, computing each wavefront vector in parallel is possible and a simple OMP pragma can do the job (implemented in WFA2lib). However, it is not effective unless each wavefront has enough length (i.e., workload) to pay off the parallelization. More sophisticated and effective intra-sequence parallelization strategies are possible (WIP).
• Intra-sequence vectorization: Yes. The core of the WFA algorithm can be automatically vectorized/SIMDed.

You are currently only offering penalties (negative scores) and the match-"score" is fixed to 0. Is this a limitation of the algorithm, or would non-negative scores work as well? I think that would be a requirement to get Protein scoring matrixes to work?

The current implementation (master branch) has this implementation. Nonetheless, the code can be adapted to match-"score" != 0 by applying a simple scores transformation formula (proposed by @jeizenga). Moreover, nothing prevents the formulation from having a different M[a,b]<0 (a!=b) cost for each substitution. Now, I have to double-check how to extend (if possible) @jeizenga's formulas to M[a,b]>0 (probably, he already has a solution for this).

I can elaborate if you want; happy to discuss!
Let us know.

[Hannes Hauswedell]

Wow, thank you for the very thorough reply!

The benchmark branch incorporates SeqAn into the align-benchmark tool, together with other alignment methods, including all WFA variations.

I haven't had luck building the branch:

gcc -Wall -g -O3 -march=native -flto -I../.. -I../../../.. -c benchmark_blockaligner.c -o ../../build/benchmark_blockaligner.o
benchmark_blockaligner.c:42:5: error: unknown type name ‘Cigar’; did you mean ‘cigar_t’?
   42 |     Cigar* const cigar,
      |     ^~~~~
      |     cigar_t

Now, I have to double-check how to extend (if possible) @jeizenga's formulas to M[a,b]>0 (probably, he already has a solution for this).

Interesting, I will also have a look at the algorithm in more detail and think about it.

Background: I am one of the original authors of SeqAn, but have recently started https://github.com/biocpp as a new project (with initial parts as a fork of SeqAn3). I am currently thinking about the BioC++ alignment library and looking for existing code that could serve as a backend. There seem to be some things that are not possible with this library (e.g. local alignment), but it would be great if other things like scoring matrixes and non-negative scores would work!

P.S: we can also move this to e-mail, maybe that is better! → [e-mail]

[Santiago Marco-Sola]

I haven't had luck building the branch:

Right. Some of the external tools are quite experimental and someone can have a hard time compiling them all together.
I know it is not much of a help, but I tried again to clone+compile on my laptop (Ubuntu 18.04, gcc 8.3.0) to double-check.
In the worst case, if block_aligner is failing, you can always comment the code on benchmark_blockaligner.c and carry on.

Background: I am one of the original authors of SeqAn, but have recently started https://github.com/biocpp as a new project (with initial parts as a fork of SeqAn3). I am currently thinking about the BioC++ alignment library and looking for existing code that could serve as a backend.

Yes, I have always liked very much the SeqAn project. It is great that the developers of such nice project look into the WFA. Our idea is to keep on extending the WFA algorithm & variants to see where it can be of use or improve current methods/implementations. On that regard, we are very happy to listen to your suggestions.