Incorrect RH Orthographic Matrix
rnvannatta opened this issue · comments
The right handed orthographic matrices do not work as expected.
They should return depth between 0 and 1 for z values between znear and zfar, like the left handed ones do, but they do not.
The issue traces to DirectXMath/Inc/DirectXMathMatrix.inl commit 7c30ba5, lines 2826 and 3014. They assign the 4th row 3rd column element to fRange * fNear
but I believe the correct value is - fRange * fFar
.
For explicitness, I believe the full correct term for M_33 is 1 / (near - far)
and the full term for M_43 is -far / (near - far)
I confirmed this value by applying the * 0.5 + 0.5
matrix to the RH orthographic matrix taken from opengl superbible 7e.
The implementation of XMMatrixOrthographicRH
has been the same since the original release. The code has been modified in various ways, but functionally it's been unchanged since the original "xboxmath". I'll have to dig into this a bit...
The implementation of XMMatrixOrthographicRH
uses:
float fRange = 1.0f / (NearZ - FarZ);
M.m[2][2] = fRange;
M.m[3][2] = fRange * NearZ;
The implementation of XMMatrixOrthographicLH
uses:
float fRange = 1.0f / (FarZ - NearZ);
M.m[2][2] = fRange;
M.m[3][2] = -fRange * NearZ;
Otherwise the values are the same.
Legacy D3DXMath used for D3DXMatrixOrthoRH
:
pOut->_33 = 1.0f / (zn - zf);
pOut->_43 = pOut->_33 * zn;
for D3DXMatrixOrthoLH
:
pOut->_33 = 1.0f / (zf - zn);
pOut->_43 = -pOut->_33 * zn;