Wrong sign in penalized optimization during warmup?
chriselrod opened this issue · comments
Chris Elrod commented
https://github.com/tpapp/DynamicHMC.jl/blob/master/src/mcmc.jl#L141
The code reads:
magnitude_penalty::T = 1e-4
...
fg! = function(F, G, q)
ℓq, ∇ℓq = logdensity_and_gradient(ℓ, q)
if G ≠ nothing
@. G = -∇ℓq - q * magnitude_penalty
end
-ℓq - (0.5 * magnitude_penalty * sum(abs2, q))
end
It looks like you're adding 0.5 * magnitude_penalty * sum(abs2, q)
to the log likelihood, instead of subtracting it. The latter would penalize extreme values of the parameters, while the former favors them.
Is this intended behavior (eg, to try and encourage more diffuse starting values)?
Tamas K. Papp commented
No, I think this is a bug. Good catch! I will think about it and fix it soon.