Many objects in our daily life are rigid, but some are non rigid and deformable. For instance, humans are deformable as we can change our extrinsic shape by moving our limbs.
Skinning is a technique to deform 3D objects using bones.
When we want to move the hand of a mesh of a human, it would be hard for us to do so by moving each and every vertex in the hand. Instead, we move a bone that affects all the vertices that are closer to it.
Implementing Skinning took me well over a week, and it wasn’t a very smooth ride. The first issue was that I had several conventions on different libraries and APIs I used.
I had one convention for the math library(http://clb.demon.fi/MathGeoLib/) another for the 3D file importer(http://assimp.sourceforge.net/) and a third of the DX shaders. They didn’t necessarily coincidence.
I admit I didn’t have the brains or didn’t properly solve this and figured it out analytically. Instead I just debugged the results and tried to fix them until it looked right.
The issue I had is that the meshes were loaded mirrored over the z-axis. There was another issue that the math lib was generating matrices of row major instead of column major, but I already knew that.
It might be a bit overwhelming to try to find how to fix this when you have a complex mesh that looks really broken. So instead you can simply use a simpler mesh with only two bones to test things out. Makes it a lot easier to debug it.
Every bone in the skinned mesh has its own resting position. A position where deformation does not occur by this specific bone. If you move a bone from that resting position, it will deform the mesh.
Mirroring the z value of each vertex did not solve the issue, I also had to mirror the offset matrix(the translation of the bone from the resting position) and that is what took me quite some time and debugging.
Now I had my bones and mesh properly match the DirectX shaders convention, but I still needed weights for the vertices. Each vertex in a skinned mesh have up to 4 weights which represent how much(up to 4) bones affect it. The weights from the modeling program were not very good and it is quite a tedious work to set the weights for each bone manually in the modeling program. What I did instead is to set the weights automatically according to the bones placement in the mesh.
The automatic weight calculation is done by turning the mesh into voxel, or voxelization. This give us the volume of the mesh up to a certain accuracy on which we can make calculations. I used marching cubes to calculate the geodesic distance of each vertex in the mesh from each bone.
This makes sense because we don’t want to give weight to the vertices according to the Euclidean(flight distance) from the bone, but rather the shortest distance within the mesh bounds.
One issue that arose was that the voxelization was really slow. It took me ten minutes to voxelize a single mesh. The reason was that I did a naive implementation. For each voxel, I checked whether it’s inside or outside the mesh, which means I had to test every voxel against the entire mesh.
The optimization I did was some variant of Octrees. I simply had a coarse 3D grid. For each cube in the grid I checked which triangles of the mesh intersect the cube. Now in order to test if a voxel is inside the mesh, I only had to test it on the triangles which a ray from the voxel intersects with the cubes that contain those triangles.
This improved the times from ten minutes to a few seconds.
If you would like more detailed explanation of skinning and voxelization, please reply in the comments, in the mean time here are the results.