The all_objects.blend test scene (in subversion tests repo) contained an
object with a subdivision surface. Which changes vertex positions
slightly, depending on used OpenSubDiv version and the compile flags. It
seems that the intent of the test was "test export of meshes that use
modifiers", so I changed that object to be a cube with a simple "taper"
modifier instead.
While at it, changed OBJ exporter test code to always print the
"expected and what we got" text difference details, when a test fails.
Much easier to see than just "the files are different" output. The code
to print that was behind an off by default flag for some reason.
This diff should get comitted together with updated all_objects templates
in subversion tests repo.
Reviewed By: Sebastian Parborg
Differential Revision: https://developer.blender.org/D14597
This commit furthers some of the changes that were started in
rBb9febb54a492 and subsequent commits by changing the way surface
objects are presented to render engines and other users of evaluated
objects in the same way. Instead of presenting evaluated surface objects
as an `OB_SURF` object with an evaluated mesh, `OB_SURF` objects
can now have an evaluated geometry set, which uses the same system
as other object types to deal with multi-type evaluated data.
This clarification makes it more obvious that lots of code that dealt
with the `DispList` type isn't used. It wasn't before either, now it's
just *by design*. Over 1100 lines can be removed. The legacy curve
draw cache code is much simpler now too. The idea behind the further
removal of `DispList` is that it's better to focus optimization efforts
on a single mesh data structure.
One expected functional change is that the evaluated mesh from surface
objects can now be used in geometry nodes with the object info node.
Cycles and the OBJ IO tests had to be tweaked to avoid using evaluated
surface objects instead of the newly exposed mesh objects.
Differential Revision: https://developer.blender.org/D14550
This takes state of soc-2020-io-performance branch as it was at
e9bbfd0c8c7 (2021 Oct 31), merges latest master (2022 Apr 4),
adds a bunch of tests, and fixes a bunch of stuff found by said
tests. The fixes are detailed in the differential.
Timings on my machine (Windows, VS2022 release build, AMD Ryzen
5950X 32 threads):
- Rungholt minecraft level (269MB file, 1 mesh): 54.2s -> 14.2s
(memory usage: 7.0GB -> 1.9GB).
- Blender 3.0 splash scene: "I waited for 90 minutes and gave up"
-> 109s. Now, this time is not great, but at least 20% of the
time is spent assigning unique names for the imported objects
(the scene has 24 thousand objects). This is not specific to obj
importer, but rather a general issue across blender overall.
Test suite file updates done in Subversion tests repository.
Reviewed By: @howardt, @sybren
Differential Revision: https://developer.blender.org/D13958
Original report (T96763) only reported the issue of double-space before the texture path, but while adding test coverage I found some other issues that I fixed while at it:
- Incorrectly emits two spaces between `map_Xx` keyword and the texture path, leading to some 3rd party software not finding the textures,
- Emissive texture map (`map_Ke`) was not exported,
- When Mapping node is used on the texture UVs, the "Location" and "Scale" values were mixed up (location written as "scale", scale written as "location).
Added gtest coverage.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D14519
On Windows/MSVC this gives a minor (~20%) speedup presumably due to a faster float/int formatter. On macOS (Xcode13), this gives a massive speedup, since snprintf that is in system libraries ends up spending almost all the time inside some locale-related mutex lock.
The actual exporter code becomes quite a bit smaller too, since it does not have to do any juggling to support std::string arguments, and the buffer handling code is smaller as well.
Windows (VS2022 release build, Ryzen 5950X 32 threads) timings:
- Blender 3.0 splash scene (2.4GB obj): 4.57s -> 3.86s
- Monkey subdivided level 6 (330MB obj): 1.10s -> 0.99s
macOS (Xcode 13 release build, Apple M1Max) timings:
- Blender 3.0 splash scene (2.4GB obj): 21.03s -> 5.52s
- Monkey subdivided level 6 (330MB obj): 3.28s -> 1.20s
Linux (ThreadRipper 3960X 48 threads) timings:
- Blender 3.0 splash scene (2.4GB obj): 10.10s -> 4.40s
- Monkey subdivided level 6 (330MB obj): 2.16s -> 1.37s
The produced obj/mtl files are identical to before.
Reviewed By: Howard Trickey, Dalai Felinto
Differential Revision: https://developer.blender.org/D13998
This change makes it possible to add implementation of common
C++ methods for DNA structures which helps ensuring unsafe
operations like shallow copy are done explicitly.
For example, creating a shallow copy used to be:
Object temp_object = *input_object;
In the C++ context it was seen like the temp_object is
properly decoupled from the input object, while in the
reality is it not. Now this code becomes:
Object temp_object = blender:🧬:shallow_copy(*input_object);
The copy and move constructor and assignment operators are
now explicitly disabled.
Other than a more explicit resource management this change
also solves a lot of warnings generated by the implicitly
defined copy constructors w.r.t dealing with deprecated fields.
These warnings were generated by Apple Clang when a shallow
object copy was created via implicitly defined copy constructor.
In order to enable C++ methods for DNA structures a newly added
macro `DNA_DEFINE_CXX_METHODS()` is to be used:
tpyedef struct Object {
DNA_DEFINE_CXX_METHODS(Object)
...
} Object;
For the shallow copy use `blender:🧬:shallow_copy()`.
The implementation of the memcpy is hidden via an internal DNA
function to avoid pulling `string.h` into every DNA header.
This means that the solution does not affect on the headers
dependencies.
---
Ideally `DNA_shallow_copy` would be defined in a more explicit
header, but don;t think we have a suitable one already. Maybe
we can introduce `DNA_access.h` ?
Differential Revision: https://developer.blender.org/D14427
This reverts commit 8c44793228.
Apparently, this generated a lot of warnings in GCC.
Didn't find a quick solution and is it not something I want to be
trading between (more quiet Clang in an expense of less quiet GCC).
Will re-iterate on the patch are re-commit it.
This change makes it possible to add implementation of common
C++ methods for DNA structures which helps ensuring unsafe
operations like shallow copy are done explicitly.
For example, creating a shallow copy used to be:
Object temp_object = *input_object;
In the C++ context it was seen like the temp_object is
properly decoupled from the input object, while in the
reality is it not. Now this code becomes:
Object temp_object = blender:🧬:shallow_copy(*input_object);
The copy and move constructor and assignment operators are
now explicitly disabled.
Other than a more explicit resource management this change
also solves a lot of warnings generated by the implicitly
defined copy constructors w.r.t dealing with deprecated fields.
These warnings were generated by Apple Clang when a shallow
object copy was created via implicitly defined copy constructor.
In order to enable C++ methods for DNA structures a newly added
macro `DNA_DEFINE_CXX_METHODS()` is to be used:
tpyedef struct Object {
DNA_DEFINE_CXX_METHODS(Object)
...
} Object;
For the shallow copy use `blender:🧬:shallow_copy()`.
The implementation of the memcpy is hidden via an internal DNA
function to avoid pulling `string.h` into every DNA header.
This means that the solution does not affect on the headers
dependencies.
---
Ideally `DNA_shallow_copy` would be defined in a more explicit
header, but don;t think we have a suitable one already. Maybe
we can introduce `DNA_access.h` ?
Differential Revision: https://developer.blender.org/D14427
Currently there is a "calc_face_normal" argument to mesh to bmesh
conversion, but vertex normals had always implicitly inherited whatever
dirty state the mesh input's vertex normals were in. Probably they were
most often assumed to not be dirty, but this was never really correct in
the general case.
Ever since the refactor to move vertex normals out of mesh vertices,
cfa53e0fbe, the copying logic has been explicit: copy the
normals when they are not dirty. But it turns out that more control is
needed, and sometimes normals should be calculated for the resulting
BMesh.
This commit adds an option to the conversion to calculate vertex
normals, true by default. In almost all places except the decimate
and edge split modifiers, I just copied the value of the
"calc_face_normals" argument.
Differential Revision: https://developer.blender.org/D14406
Old python exporter in 3.0 and earlier ordered faces by material,
but the new C++ exporter in 3.1+ did not, and was just writing them
in whatever is the order of the mesh data structure.
This mostly does not cause problems, except in some apps e.g.
Procreate -- for large enough meshes, this lack of
"order by material" (which ends up having more usemtl lines)
ends up creating more mesh subsets than necessary inside Procreate.
The change is not computationally heavy, e.g. exporting 6-level
subdivided Monkey mesh goes 1085ms -> 1105ms on my machine.
Reviewed By: @howardt
Differential Revision: https://developer.blender.org/D14368
This is patch D14349 from Aras Pranckevicius.
The logic in the code was _completely different_ from the documentation
and what the python exporter in 3.0 did. The new code assumed that
"export material groups" meant "append material name to the object name",
and was only ever kicking in when the "export object groups" option was
also checked. But the proper behavior (as in 3.0 exporter & the online docs),
is to emit g objectname_materialname before each usemtl line. Which is something entirely else.
This is patch D14347 from Aras Pranckevicius.
Instead of scaling "the scene" (i.e. transform vertices by object matrix,
then multiply by scale factor), it was instead first applying the scale
factor in local space, and then transforming by the object matrix.
This applies patch D14343 from Aras Pranckevicius, with a description:
The new 3.1+ OBJ exporter did not have correct logic when faced with
non-uniform & mirrored (negative on odd number of axes) object scale:
- Normals were not transformed correctly (should use inverse transpose of the matrix),
and were not normalized,
- Face order was not "flipped" when transform has negative scale on odd number of axes
(visible when using "face orientation" viewport overlay).
This commit renames enums related the "Curve" object type and ID type
to add `_LEGACY` to the end. The idea is to make our aspirations clearer
in the code and to avoid ambiguities between `CURVE` and `CURVES`.
Ref T95355
To summarize for the record, the plans are:
- In the short/medium term, replace the `Curve` object data type with
`Curves`
- In the longer term (no immediate plans), use a proper data block for
3D text and surfaces.
Differential Revision: https://developer.blender.org/D14114
This patch reverses the dependency between `BLI_math_vec_types.hh` and
`BLI_math_vector.hh`. Now the higher level `blender::math` functions
depend on the header that defines the types they work with, rather than
the other way around.
The initial goal was to allow defining an `enable_if` in the types header
and using it in the math header. But I also think this operations to types
dependency is more natural anyway.
This required changing the includes some files used from the type
header to the math implementation header. I took that change a bit
further removing the C vector math header from the C++ header;
I think that helps to make the transition between the two systems
clearer.
Differential Revision: https://developer.blender.org/D14112
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Fixes T95384. New exporter was missing a fix for T94516 that recently got applied to the python exporter.
Also changed the obj export tests code so that when save_failing_test_output is requested and MTL result is different from the golden expectation, it is saved as well, similar to how it's done for the OBJ file result.
This change from Aras further parallelizes wihin large meshes (the previous one
just parallelized over objects).
Some stats: on A Windows machine, AMD Ryzen (32 threads):
(one mesh) Monkey subdivided to level 6: 4.9s -> 1.2s (blender 3.1 was 6.3s; 3.0 was 49.4s).
(one mesh) "Rungholt" minecraft level: 8.5s -> 2.9s (3.1 was 10.5s; 3.0 was 73.7s).
(lots of meshes) Blender 3 splash: 6.2s -> 5.2s (3.1 was 48.9s; 3.0 was 392.3s).
On a Linux machine (Threadripper, 48 threads, writing to SSD):
Monkey - 5.08s -> 1.18s (4.2x speedup)
Rungholt - 9.52s -> 3.22s (2.95x speedup)
Blender 3 splash - 5.91s -> 4.61s (1.28x speedup)
For details see patch D14028.
Also fixed conflicts due to the change in file writing in the new obj exporter
in master, and fixed one of the tests that was added in master but not 3.1.
The new wavefront .obj exporter in 3.1 was producing slightly invalid parm line syntax (missing u), and was not setting first/last N params to zeroes and ones for curves with "endpoint" flag properly.
This is a patch from Aras Pranckevicius, D13927. See that patch for full
details. On Windows, the many small fprintfs were taking up a large amount
of time and significant speedup comes from using snprintf into chained buffers,
and writing them all out later.
On both Windows and Linux, parallelizing the processing by Object can also lead
to a significant increase in speed.
The 3.0 splash screen scene exports 8 times faster than the current C++ exporter
on a Windows machine with 32 threads, and 5.8 times faster on a Linux machine
with 48 threads.
There is admittedly more memory usage for this, but it is still using 25 times
less memory than the old python exporter on the 3.0 splash screen scene, so
this seems an acceptable tradeoff. If use cases come up for exporting obj files
that exceed the memory size of users, a flag could be added to not parallelize
and write the buffers out every so often.
Previously, the new obj exporter was only exporting per-vertex normals for faces
marked as "smooth". But a face can have custom normals, as soon as the normals
data layer exists. This change makes it follow the behavior of USD & Collada
exporters and the old Python one, which also export per-vertex normals as soon
as the layer is there. (From Patch D13957.)
The new OBJ exporter did not handle object instances.
The fix is to use a dependency graph iterator, asking for instances.
Unfortunately that iterator makes a temporary copy of instance objects
that does not persist past the iteration, but we need to save all the
objects and meshes to write later, so the Object has to be copied now.
This changed some unit tests. Even though the tests don't have instancing,
the iterator also picks up some Text objects as Mesh ones (which is a good
thing), resulting in two more objects in the all_objects.obj file output.
Some new obj exporter tests were disabled because the normals were different
in the last decimal place on different platforms.
The old python exporter deduped normals with their coordinates rounded to
four decimal places. This change does the same in the new exporter.
On one test, this produced a file 25% smaller and even ran 10% faster.
The switch to how normals are kept has led to tiny differences in
the normal output values on different platforms. Disabling the failing
tests while working on a solution to this problem.
As described in T91186, this commit moves mesh vertex normals into a
contiguous array of float vectors in a custom data layer, how face
normals are currently stored.
The main interface is documented in `BKE_mesh.h`. Vertex and face
normals are now calculated on-demand and cached, retrieved with an
"ensure" function. Since the logical state of a mesh is now "has
normals when necessary", they can be retrieved from a `const` mesh.
The goal is to use on-demand calculation for all derived data, but
leave room for eager calculation for performance purposes (modifier
evaluation is threaded, but viewport data generation is not).
**Benefits**
This moves us closer to a SoA approach rather than the current AoS
paradigm. Accessing a contiguous `float3` is much more efficient than
retrieving data from a larger struct. The memory requirements for
accessing only normals or vertex locations are smaller, and at the
cost of more memory usage for just normals, they now don't have to
be converted between float and short, which also simplifies code
In the future, the remaining items can be removed from `MVert`,
leaving only `float3`, which has similar benefits (see T93602).
Removing the combination of derived and original data makes it
conceptually simpler to only calculate normals when necessary.
This is especially important now that we have more opportunities
for temporary meshes in geometry nodes.
**Performance**
In addition to the theoretical future performance improvements by
making `MVert == float3`, I've done some basic performance testing
on this patch directly. The data is fairly rough, but it gives an idea
about where things stand generally.
- Mesh line primitive 4m Verts: 1.16x faster (36 -> 31 ms),
showing that accessing just `MVert` is now more efficient.
- Spring Splash Screen: 1.03-1.06 -> 1.06-1.11 FPS, a very slight
change that at least shows there is no regression.
- Sprite Fright Snail Smoosh: 3.30-3.40 -> 3.42-3.50 FPS, a small
but observable speedup.
- Set Position Node with Scaled Normal: 1.36x faster (53 -> 39 ms),
shows that using normals in geometry nodes is faster.
- Normal Calculation 1.6m Vert Cube: 1.19x faster (25 -> 21 ms),
shows that calculating normals is slightly faster now.
- File Size of 1.6m Vert Cube: 1.03x smaller (214.7 -> 208.4 MB),
Normals are not saved in files, which can help with large meshes.
As for memory usage, it may be slightly more in some cases, but
I didn't observe any difference in the production files I tested.
**Tests**
Some modifiers and cycles test results need to be updated with this
commit, for two reasons:
- The subdivision surface modifier is not responsible for calculating
normals anymore. In master, the modifier creates different normals
than the result of the `Mesh` normal calculation, so this is a bug
fix.
- There are small differences in the results of some modifiers that
use normals because they are not converted to and from `short`
anymore.
**Future improvements**
- Remove `ModifierTypeInfo::dependsOnNormals`. Code in each modifier
already retrieves normals if they are needed anyway.
- Copy normals as part of a better CoW system for attributes.
- Make more areas use lazy instead of eager normal calculation.
- Remove `BKE_mesh_normals_tag_dirty` in more places since that is
now the default state of a new mesh.
- Possibly apply a similar change to derived face corner normals.
Differential Revision: https://developer.blender.org/D12770
