A description of the problem is found in https://developer.blender.org/T36425#19 .
The cause of the issue was identified as roudning errors in Operators::createStroke() due
to insufficient numerical precision. Precision promotion from float to double was done in
the return values of getPoint3D/2D methods in Interface0D and its subclasses in C++
(data members stored in the 0D classes have already been in double precision).
before entering the recursion check.
Now use group tags instead of object tags, which could be a little more
efficient and was used before this patch too.
It still warns the user that there may be an error, but the baking goes
on. Also using the new is_uniform_scaled_m4() instead of float comparison.
Reported and fix suggested by Campbell Barton as a concern over 2bfc3deb
The scene file provided by the problem report has many degenerate faces coming from
a particle system. These zero-area faces were not expected in the ray-casting line visibility
algorithms of Freestyle. Now degenerate faces are properly excluded from the imported
mesh data and not fed to the line visibility algorithms.
The new recursion check for groups duplicating themselves has to un-set the LIB_DOIT flag after each object, otherwise it will prevent duplicating a (non-recursive) group multiple times.
To get the versioning code properly work, the condition should have been:
DNA_struct_elem_find(fd->filesdna, "FreestyleLineStyle", "MTex", "*mtex[18]")
The present commit uses another new structure member instead, to avoid referring
to the magic number 18 in the last string literal.
This was because to the filtering code, those FCurves still weren't in any
groups, and so couldn't be visible (since a temporary group is created to
house them). As a result, the visible-channels list would be empty, causing
all hidden FCurves to be treated as hidden.
These were broken by 1f3655d224, since
an argument of the wrong type was getting passed to ANIM_animdata_filter(),
resulting in no channels ever being picked up for the "visible channels" list.
Turned out there was still quite a few cases were indices were set dirty,
but elem_index_dirty was not tagged accordingly (mostly for BM_LOOP,
but a few others as well). So probably this crash was not the only one
hidden here.
Hopefully all possible cases were catched this time!
This fixes most of the cases, the only situation not addressed is when
the highpoly object(s) has non-uniform scale.
mul_transposed_mat3_m4_v3() should take care of non-uniform scales so
I'm a bit confused on why it doesn't work. The lowpoly object can have
any transformation, the only issue is if the highpoly object has
non-uniform scale.
Test file of the remaining issue:
https://developer.blender.org/file/info/PHID-FILE-tpw2xgddyzxtpg3e7xzs/
Reference reading:
http://www.unknownroad.com/rtfm/graphics/rt_normals.html
bleeding at the symmetry edges.
We need an extra way to detect if faces overlap here. An easy way is to
detect the winding of the faces in UV space. If the winding differs, the
faces will naturally overlap. I have tried a few approaches here such as
choosing an offset point from the middle of the edge for intersection in
both faces of the edge in uv space, but winding is the safest way and
should work with very small faces/dense meshes too.