The issue was that the code tried to use the bones transformation matrix as a rotation matrix. This works fine as long as the scale is 1. Now we simply make sure that we only get a pure rotation matrix when extracting it for the bone's transformation matrix.