xiph / daala

Modern video compression for the internet

Home Page:https://xiph.org/daala/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Integer overflow in test_coef_coder

tmatth opened this issue · comments

Due to commit 9c83727 there is an integer overflow.

src/tests/../generic_encoder.c:105:12: runtime error: signed integer overflow: 2 * 1091604154 cannot be represented in type 'int'

To reproduce do ./ci/clang-ioc.sh and run ./test_coef_coder

@jmvalin, can
e = ((2**ex_q16 >> 8) + (1 << shift >> 1)) >> shift;
be converted to
e = ((*ex_q16 >> 7) + (1 << shift >> 1)) >> shift;
or
e = ((2*(*ex_q16 >> 8)) + (1 << shift >> 1)) >> shift;