Curse you, D3D9 (aka depth shadowmapping precision solved)

· by Steve · Read in about 2 min · (392 Words)

Just a quick follow-up, I mused in my earlier post today that the fact that D3D9 inexplicably gives you an integer format texture (D3DFMT_G16R16) when you ask for a 16-bit floating point texture (D3DFMT_R16F), was the root of my precision problems in depth shadowmapping, and indeed that was the case. I also found another contributing error (a slipup in the clipspace-to-texturespace matrix) but D3D’s odd behaviour had a significant influence.

I can understand that the card wants to give you a 32-bits per pixel format for alignment purposes, I can’t understand why it chooses to turn it into a non-floating point format. If it was going to align it, I would have expected a D3DFMT_G16R16F which is the same size but each channel is floating point. The integer format I was being given just didn’t have the variable range required for depth comparisons. Whilst some depth shadowmapping systems use large integer textures, to do so requires a re-scaling of the scene depth range into the range of the integer, which while possible (OGRE has bindings for that), doing it highly complicates the use of depth bias if the scene range is variable - which for anything except a simple fixed-scene demo, it always is.

Anyway, I have it working properly now using R32F, and it’s actually no slower anyway (same fillrate burn I guess). So be advised, whilst it’s useful to use half-float formats for RGB/RGBA since it does save bandwidth (I certainly saw this in my HDR implementation, which is why I started on half-float formats here), single-channel half-float textures just aren’t worth it. You can request the 2-channel green/red ones now anyway so that’s a good fallback and lets you pack 2 half-float values into 32 bits. I can’t believe how much time I wasted farting around with my biasing calculations, thinking I was doing something wrong, when in fact this was the cause. I had planned to implement the GLSL version with the HLSL version still broken to see if it worked, and I think it would have done (the integer conversion being a D3D thing I expect), but I wouldn’t have known why. Thank goodness for the lock format error I had this morning which highlighted this whole area to me, otherwise I could have spent another couple of days swearing and bashing my head against the monitor.