marian-nmt / marian

Fast Neural Machine Translation in C++

Home Page:https://marian-nmt.github.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

negative cost with negative weights

tomsbergmanis opened this issue · comments

The documentation says that word weights can be real-valued. Real numbers can be negative. This is convenient because outputs of models one could use for scoring often output log-likelihoods, which are negative numbers.
image
However, using negative weights results in a negative cost value, which, if minimized, maximizes the error:
image

  • If this is a mistake in the documentation, could we update the documentation to say "non-negative numbers"?
  • If not, could we update the documentation with the steps to be taken to avoid divergence with negative weights?

Cheers,
Toms