Weighted logsumexp
zsteve opened this issue · comments
Let w, x be vectors. I'm interested in computing log(dot(w, exp.(x)). The current logsumexp functions can be used to compute this, but one must take log.(w)which is troublesome when w can be very small or zero. I took a look at the source and I think actually it should be straightforward to extend the current code. The end goal for this would be to implement x -> log.(K*exp.(x)) where K is a matrix and x is a vector.
What I'm not so familiar with is how to implement the reduction in Julia in an efficient manner. At a high level, the first thing that comes to mind for me is to do something like a binary operation of the type x, (y, w) -> log(exp(x) + w*exp(y)) and do a reduction over pairs of data and weights. But I'd appreciate some input on how this should be best done.
There's a PR in the StatsFuns repo that implements weighted logsumexp. IIRC it was opened before this package was extracted from StatsFuns and the implementation of logsumexp was rewritten. There's also a discussion of a weighted alternative of the single-pass algorithm of logsumexp in LogExpFunctions in https://mileslucas.com/posts/weighted-logsumexp/.