str_fastcmp uses uninitialized bytes for its result

[Bernhard M. Wiedemann]

Because it fetches and processes 4 bytes and 1-3 of these can be behind the end of string containing uninitialized random data.

valgrind --tool=memcheck luajit -bg bcc.lua bcc.o
showed

Conditional jump or move depends on uninitialised value(s)
   at 0x1636B2: UnknownInlinedFun (lj_str.c:48)
   by 0x1636B2: lj_str_new (lj_str.c:179)
   by 0x1655DA: lua_pushlstring (lj_api.c:688)
   by 0x14EDEE: emptybuffer (lib_aux.c:202)
   by 0x14F288: luaL_pushresult (lib_aux.c:253)
   by 0x1272B7: lj_cf_string_gsub (lib_string.c:654)
   by 0x169799: lj_BC_FUNCC (buildvm_x86.dasc:811)
   by 0x16020F: lua_pcall (lj_api.c:1197)
   by 0x167A46: UnknownInlinedFun (luajit.c:392)
   by 0x167A46: UnknownInlinedFun (luajit.c:486)
   by 0x167A46: pmain (luajit.c:551)

Seen with moonjit-2.2.0 on openSUSE

This bug has been found while working on reproducible builds for openSUSE.

[Thomas Fransham]

There is an existing valgrind suppression file at src/lj.supp for this false positive.

[Bernhard M. Wiedemann]

It seems this was added in commit 7991a66
but that does not explain why it would be a false positive.

I carefully looked at the code and still think that it indeed can use uninitialized bytes and thus report identical strings as different, unless every string was followed by 3 or 4 zero-bytes. But then valgrind wouldnt be reporting this.

https://github.com/moonjit/moonjit/blob/master/src/lj_str.c#L46

As an example, try to think of a call to str_fastcmp("a", "a", 1)

That would fetch the first 4 bytes of both strings in line 47 and XOR them.
But only 2 of the 4 fetched bytes are defined, so it is likely that the test in line 48
would find that v is not 0. Thus it would return one as larger than the other, depending on what random bytes still were in the memory behind the 2 defined bytes.
Or maybe the shifting in line 51 is meant to compensate that?
Ah, indeed it seems that it can also return 0 in line 51 but that also means that unlike strcmp, it would not tell which string is "larger".

[Siddhesh Poyarekar]

It looks like you've found by yourself at the reason as to why it doesn't actually 'use' the uninitialized bytes; FWIW, the str* variants in glibc also do something similar (and have page end checks and all that) to avoid misaligned accesses. To get the correct unequal result, it does some extra computation on top to get the right return value. I suspect that either valgrind knows about this and supresses the warnings for glibc or there's some property of the glibc accesses that valgrind is OK with. We'll know when we compare the generated assembly.

We don't need the correct unequal value for str_fastcmp though; we just want to know if they're equal or not and the function does the job. It can be rewritten to be nicer to read and perhaps even intuitive enough for valgrind to understand that it's safe. Lets continue this discussion on your PR on the right approach to resolve this.