The Radeon X1x00 range supposedly supports Shader Model 3. Except of course, that it doesn’t. 😉 ATI have exploited a loophole in the DirectX9 SM3 definition that says to be SM3 compliant, you have to implement vertex texture fetch. However, it gives manufacturers flexibility over which texture formats it can support in the vertex pipeline (nVidia support PF_FLOAT32_R and PF_FLOAT32_RGBA in Ogre pixelformat conventions), and what it doesn’t say is that you have to support at least one texture format as a vertex texturing source. Guess what? The X1x00 supports a total of 0 texture formats for vertex texturing. But it’s still technically SM3 compliant because their drivers respond to the request to bind vertex textures, you just can’t actually use it for anything. Sneaky.
This means vertex texture fetching in the conventional, standardised form is impossible on ATI cards at the moment, despite this feature making it into the XBox 360’s ATI chip. ATI will tell you that instead they support the ‘more flexible’ Render-to-vertex-buffer approach (R2VB), where you use a separate pass to render to a texture, then reinterpret this buffer area as a vertex stream. This, of course, is one of those things that’s earmarked for DirectX10.
But there are 2 problems with that assertion - firstly DirectX9 doesn’t support R2VB, so they’ve made it possible through a hideous, makes-your-eyes-bleed sort of hack where they appear to have used an unused part of the point size render state to enable R2VB, along with a dummy vertex stream binding. It’s really, really nasty. Secondly, whilst R2VB is really handy if that’s what you want to do (dynamic vertex effects driven entirely on the GPU), but if all you want to do is feed in a static texture, it’s a pain in the ass because it forces you into a 2-pass solution where you have to pass your texture through one RTT pass, then feed the result back again through a vertex stream. You also can’t do anything clever with dynamic texture coords without repeating this 2-pass process.
Bottom line is that true vertex texturing was a required part of SM3, and ATI wriggled out of it on a technicality, kludging a feature their architechture happened to support more readily to take its place, but it’s like hammering a square peg into a round hole. This kludge only works in DirectX, too.
I deliberated over whether to support the R2VB hack in Dx9 and decided against it. For a start, it’s a fundamentally different approach and so building a system where you could have a proper VTF technique for nVidia, and an automated fallback R2VB technique for ATI, was pretty much impossible, there’s just too much difference in the top-level approach. Plus, it would only work on Windows anyway.
So, SM3 vertex texturing will be an nVidia-only domain in Ogre until ATI implement it properly, which will probably happen with their next generation of cards. This isn’t an nVidia bias, supporting VTF over R2VB, it’s a standards position - VTF is standardised, R2VB is not (yet), it’s ATI’s fault for not sticking to the standard. Plus, ATI’s R2VB implementation is just too invasive ahead of the Dx10 standardisation process to be useful to most people. After Dx10 it will be more useful since it can be relied upon.