[0017] Investigate catastrophic failures more cloesly
llvm-beanz opened this issue · comments
Which proposal does this relate to?
0017 Conforming Literals
Describe the issue or outstanding question.
During experimentation with existing shaders we discovered several cases with catastrophic rendering defects. We should investigate them to determine if we need to do additional mitigations.
Investigating this a bit further this is not at all what I expected. The problematic behavior change occurs with code like this:
export float Fn(int inInt, inFloat) {
int Val = (inInt & 0xffff0000) >> 16);
return Val* inFloat
}
With HLSL's literal int
type 0xffff0000
is signed, which means the bit shifts are arithmetic shifts filling in the sign bit.
define float @"\01?Fn@@YAMHM@Z"(i32 %inInt, float %inFloat) #0 {
%1 = ashr i32 %inInt, 16
%2 = sitofp i32 %1 to float
%3 = fmul fast float %2, %inFloat
ret float %3
}
With C's rules 0xffff0000
is unsigned, and the logical operators produce an unsigned result with 0's shifted in at the most significant bits.
define float @"\01?Fn@@YAMHM@Z"(i32 %inInt, float %inFloat) #0 {
%1 = lshr i32 %inInt, 16
%2 = sitofp i32 %1 to float
%3 = fmul fast float %2, %inFloat
ret float %3
}