I added morph animation to Dagon a little while back, but I’ve recently been pondering some further enhancements, mainly following a few experiments with XSI v5.0’s shape animation. The problem with animation is that there are so many variables. Firstly you can have morph animation, skeletal animation, or a combination of both. Then, either one can be powered by the CPU or by shaders. Finally, you have stencil shadows (potentially hardware extruded) adding their extra little complications. Each one of these can affect the vertex data that has to be sent to the card, with things like binding temp buffers, w-buffers (for shadows), blend weights / indices (for hardware skinning), secondary positions (for hardware morphing). I got to the stage where my head was spinning so much I had to put in some restrictions just so I could get it done.
The current situation is:
- Morph animation and skeletal animation can be used in any combination
- Both can be hardware accelerated, but in order for hardware acceleration to apply for either, both have to be implemented in your shader if you use both types of animation at once.
- Even though multiple skeletal animations can be blended, only one morph animation can be active on a SubEntity at once. This was to make hardware morphing feasible, since it requires 2n+1 vertex buffers to do (a ‘from’ and ‘to’ keyframe for each animation, plus the original data). Only allowing one animation means you only ever need 2, ie the ‘from’ and ‘to’ keyframe, if the vertex data passed is absolute
This is all fine for implemeting the kind of morph animation like the nasty old MD2 format, or for doing complete shape morphing, and the hardware friendliness of it matches OGRE’s emphasis well. However, the main problem is facial animation. Facial animation typically needs several facial ‘poses’, perhaps only affecting a subset of the face, which can be freely combined at various weights. I had been trying to incorporate this into the same approach as ‘full’ morph animation, made up of many tracks, and it became infeasible, mainly because of the excessive buffer requirements.
Now I’ve come back to this with a fresh head, I see that I just need to think of it another way. Facial animation doesn’t need tracks in the conventional sense - in fact if each track is a facial pose, then there would be only 1 keyframe in each one, to represent the change from the base pose. So unlike full blended morph animation where you would need 3 vertex values (base position, keyframe1 and keyframe2) to calculate the final position for a weighted blend, for this you only need 2 (base position, final keyframe), and the interpolation value (t) becomes the same as a weight.
Now, I think for this I need another type of track & keyframe, since there are some significant differences to the ‘full’ morph animation track/keyframe. Namely:
- Tracks can only have one keyframe, being the final pose
- The keyframe doesn’t have to include a full snapshot of the entire SubMesh, it only includes the vertices that will change. Note that when doing this in hardware though it will have to be padded out to full size since vertex shaders can’t ignore certain vertices
- The values should be offsets to the original vertex position to promote blending. Since only one keyframe is present per track the number of buffers required is n+1.
You might think that 2 and 3 could apply to full morph animation too, and you wouldn’t be wrong. But, bear in mind that since blending of multiple animations with multiple keyframes in full morph animation can’t realistically be done in hardware because of the 2n+1 vertex buffer requirement. As mentioned above we get around that by using only a single animation, and using absolute rather than offset values, using only 2 vertex buffers (using offsets would need 3). The need for subsets of vertices strongly diminishes without multiple animations too; so I expect full morph animation to remain the same.
It will be a little tricky to keep the vertex data up to date depending on the number of facial animation tracks in use at once - I’ll be using 3D texture coordinates to pass them in, just like I do already for hardware morphing. But instead of being a fixed 2, there will be n+1 buffers (the original plus n 3D texture units, one for each active facial pose). You have to bear in mind that vertex shaders using lower profiles won’t have iteration available, so when the number of poses is variable we’d have to zero out the rest to keep the shader working. Hmm, this makes me think that a ‘includes_facial_animation’ attribute on the vertex shader should include the number of supported simultaneous poses in fact. Yes, that would make more sense and it will make my code simpler too.
So, time to delve into this again. It always takes a mammoth amount of testing to make sure it all hangs together in all combinations! But worth doing I think. So Dagon will hopefully have 3 types of mesh animation, skeletal, morph and facial. The latter 2 are mutually exclusive but both will be able to be combined with skeletal.