On linux a man page is generated before the
scripts are installed, this lead to USD getting
a null pointer for it's pluging path and crashing
the process.
Make the `Object *` and `Mesh *` parameter of `USD_mesh_topology_changed()`
`const`. This function only inspects them, and doesn't need to modify
them.
No functional changes.
The executable would get boost python linking in when not needed, and even when
linking to Python libraries there were still unresolved symbols. Instead split
off boost python libraries and link them only where needed.
Shared libraries and USD plugins will be placed in the same folder, where USD
already looks for plugins.
This means that specifying the path to the plugins will no longer be needed
once the new libraries are available for all platforms. For now the code was
refactored to support both cases.
Ref T99618
These were redundant for one of a few reasons:
- A call to `BKE_mesh_tag_coords_changed` was correct instead
- A mesh has dirty normals when created from scratch anyway
- The call was redundant with `BKE_mesh_runtime_clear_geometry`
Updated the code to use the UsdGeomPrimvarsAPI class to read
and write mesh primvars instead of the now deprecated primvars
accessors in UsdGeomImageable. This will be required to build
with USD 22.11 in the future, where the deprecated functions
have been removed.
UsdImagingCapsuleAdapter static functions GetMeshPoints() and
GetTopology() have been removed in USD 22.11. In anticipation
of this upcoming change, I've updated the test code to call
the corresponding virtual functions instead.
Using varargs had the disadvantages, replace with a macro which has
some advantages.
- Arguments are type checked.
- Less verbose.
- Unintended NULL arguments would silently terminate joining paths.
- Passing in a NULL argument warns with GCC.
Part of T101073: after the view layer sync was made lazy (D15885), the
BKE_layer_collection_resync_forbid and BKE_layer_collection_resync_allow
in Alembic/USD/OBJ importers is no longer needed, as long as they do
view layer dependent operations (selecting new objects) in a separate
loop after creating all the objects.
Verified that this does not regress import times for 26k objects OBJ
scene (Blender 3.0 splash) and 250k objects USD scene (Moana).
This is the conventional way of dealing with unused arguments in C++,
since it works on all compilers.
Regex find and replace: `UNUSED\((\w+)\)` -> `/*$1*/`
This is very similar to D14077. There are two differences though.
First is that vertex creases are already stored in a separate layer,
and second is that we can now completely remove use of `Mesh.cd_flag`,
since that information is now inherent to whether the layers exist.
There are two functional differences here:
* Operators are used to add and remove layers instead of a property.
* The "crease" attribute can be created and removed by geometry nodes.
The second change should make various geometry nodes slightly faster,
since the "crease" attribute was always processed before. Creases are
now interpolated generically in the CustomData API too, which should
help maintain the values across edits better.
Meshes get an `edge_creases` RNA property like the existing vertex
property, to provide more efficient access to the data in Cycles.
One test failure is expected, where different rounding between float
the old char storage means that 5 additional points are scattered in
a geometry nodes test.
Differential Revision: https://developer.blender.org/D15927
This is already the case for most CMake usage.
Although some find modules are an exception to this, as they were
originally maintained externally they use some different conventions.
Also corrected bad indentation in: intern/cycles/CMakeLists.txt
When a change happens which invalidates view layers the syncing will be postponed until the first usage.
This will improve importing or adding many objects in a single operation/script.
`BKE_view_layer_need_resync_tag` is used to tag the view layer to be out of sync. Before accessing
`BKE_view_layer_active_base_get`, `BKE_view_layer_active_object_get`, `BKE_view_layer_active_collection`
or `BKE_view_layer_object_bases` the caller should call `BKE_view_layer_synced_ensure`.
Having two functions ensures that partial syncing could be added as smaller patches in the future. Tagging a
view layer out of sync could be replaced with a partial sync. Eventually the number of full resyncs could be
reduced. After all tagging has been replaced with partial syncs the ensure_sync could be phased out.
This patch has been added to discuss the details and consequences of the current approach. For clarity
the call to BKE_view_layer_ensure_sync is placed close to the getters.
In the future this could be placed in more strategical places to reduce the number of calls or improve
performance. Finding those strategical places isn't that clear. When multiple operations are grouped
in a single script you might want to always check for resync.
Some areas found that can be improved. This list isn't complete.
These areas aren't addressed by this patch as these changes would be hard to detect to the reviewer.
The idea is to add changes to these areas as a separate patch. It might be that the initial commit would reduce
performance compared to master, but will be fixed by the additional patches.
**Object duplication**
During object duplication the syncing is temporarily disabled. With this patch this isn't useful as when disabled
the view_layer is accessed to locate bases. This can be improved by first locating the source bases, then duplicate
and sync and locate the new bases. Will be solved in a separate patch for clarity reasons ({D15886}).
**Object add**
`BKE_object_add` not only adds a new object, but also selects and activates the new base. This requires the
view_layer to be resynced. Some callers reverse the selection and activation (See `get_new_constraint_target`).
We should make the selection and activation optional. This would make it possible to add multiple objects
without having to resync per object.
**Postpone Activate Base**
Setting the basact is done in many locations. They follow a rule as after an action find the base and set
the basact. Finding the base could require a resync. The idea is to store in the view_layer the object which
base will be set in the basact during the next sync, reducing the times resyncing needs to happen.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15885
Related to {D15885} that requires scene parameter
to be added in many places. To speed up the review process
the adding of the scene parameter was added in a separate
patch.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15930
These changes were implemented by Sonny Campbell.
Fixed the first issue by freeing the operator customdata when the import
is cancelled.
Fixed the second issue by using a character array instead of allocating
new memory for the prim_path_mask.
Differential Revision: https://developer.blender.org/D15781
- Remove "take ownership" argument which was confusing and always true
- The argument made ownership very confusing
- Better to avoid boolean arguments that switch a function's purpose
- Remove "mask" argument which was basically wrong and not used properly
- "EVERYTHING" was used because developers are wary of removing data
- Instead use `CD_MASK_MESH` for its purpose of original mesh data
- Remove use of shallow copied temporary mesh, which is unnecessary now
- Split shape key processing into separate functions and use C++ types
- Copy fields explicitly rather than using memcpy for the whole struct
- Use higher level functions and avoid redundant code
- The whole idea is pretty simple and can be built from standard logic
- Adjust `CustomData` logic to be consistent with "assign" expectations
- Clear the layer data from the source, and moves the anonymous ID
Differential Revision: https://developer.blender.org/D15857
Replace `mesh_attributes`, `mesh_attributes_for_write` and the point
cloud versions with methods on the `Mesh` and `PointCloud` types.
This makes them friendlier to use and improves readability.
Differential Revision: https://developer.blender.org/D15907
Use `verts` instead of `vertices` and `polys` instead of `polygons`
in the API added in 05952aa94d. This aligns better with
existing naming where the shorter names are much more common.
For copy-on-write, we want to share attribute arrays between meshes
where possible. Mutable pointers like `Mesh.mvert` make that difficult
by making ownership vague. They also make code more complex by adding
redundancy.
The simplest solution is just removing them and retrieving layers from
`CustomData` as needed. Similar changes have already been applied to
curves and point clouds (e9f82d3dc7, 410a6efb74). Removing use of
the pointers generally makes code more obvious and more reusable.
Mesh data is now accessed with a C++ API (`Mesh::edges()` or
`Mesh::edges_for_write()`), and a C API (`BKE_mesh_edges(mesh)`).
The CoW changes this commit makes possible are described in T95845
and T95842, and started in D14139 and D14140. The change also simplifies
the ongoing mesh struct-of-array refactors from T95965.
**RNA/Python Access Performance**
Theoretically, accessing mesh elements with the RNA API may become
slower, since the layer needs to be found on every random access.
However, overhead is already high enough that this doesn't make a
noticible differenc, and performance is actually improved in some
cases. Random access can be up to 10% faster, but other situations
might be a bit slower. Generally using `foreach_get/set` are the best
way to improve performance. See the differential revision for more
discussion about Python performance.
Cycles has been updated to use raw pointers and the internal Blender
mesh types, mostly because there is no sense in having this overhead
when it's already compiled with Blender. In my tests this roughly
halves the Cycles mesh creation time (0.19s to 0.10s for a 1 million
face grid).
Differential Revision: https://developer.blender.org/D15488
Add a dedicated `owner_id` pointer to ID types that can be embedded
(Collections and NodeTrees), and modify slightly come code to make
handling those more safe and consistent.
This implements first part of T69169.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D15838
Fixes issues in importers written in C++ (T100737):
- Materials had one reference count too much. Affected Collada,
Alembic, USD, OBJ importers, looks like "since forever".
- Active material index was not properly set on imported meshes.
Regression since 3.3 (D15145). Affected Alembic, USD, OBJ. Note:
now it sets the first material as the active one, whereas
previously the last one was set as active. First one sounds more
"intuitive" to me.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15831
This patch moves material indices from the mesh `MPoly` struct to a
generic integer attribute. The builtin material index was already
exposed in geometry nodes, but this makes it a "proper" attribute
accessible with Python and visible in the "Attributes" panel.
The goals of the refactor are code simplification and memory and
performance improvements, mainly because the attribute doesn't have
to be stored and processed if there are no materials. However, until
4.0, material indices will still be read and written in the old
format, meaning there may be a temporary increase in memory usage.
Further notes:
* Completely removing the `MPoly.mat_nr` after 4.0 may require
changes to DNA or introducing a new `MPoly` type.
* Geometry nodes regression tests didn't look at material indices,
so the change reveals a bug in the realize instances node that I fixed.
* Access to material indices from the RNA `MeshPolygon` type is slower
with this patch. The `material_index` attribute can be used instead.
* Cycles is changed to read from the attribute instead.
* BMesh isn't changed in this patch. Theoretically it could be though,
to save 2 bytes per face when less than two materials are used.
* Eventually we could use a 16 bit integer attribute type instead.
Ref T95967
Differential Revision: https://developer.blender.org/D15675
When allocating new `CustomData` layers, often we do redundant
initialization of arrays. For example, it's common that values are
allocated, set to their default value, and then set to some other
value. This is wasteful, and it negates the benefits of optimizations
to the allocator like D15082. There are two reasons for this. The
first is array-of-structs storage that makes it annoying to initialize
values manually, and the second is confusing options in the Custom Data
API. This patch addresses the latter.
The `CustomData` "alloc type" options are rearranged. Now, besides
the options that use existing layers, there are two remaining:
* `CD_SET_DEFAULT` sets the default value.
* Usually zeroes, but for colors this is white (how it was before).
* Should be used when you add the layer but don't set all values.
* `CD_CONSTRUCT` refers to the "default construct" C++ term.
* Only necessary or defined for non-trivial types like vertex groups.
* Doesn't do anything for trivial types like `int` or `float3`.
* Should be used every other time, when all values will be set.
The attribute API's `AttributeInit` types are updated as well.
To update code, replace `CD_CALLOC` with `CD_SET_DEFAULT` and
`CD_DEFAULT` with `CD_CONSTRUCT`. This doesn't cause any functional
changes yet. Follow-up commits will change to avoid initializing
new layers where the correctness is clear.
Differential Revision: https://developer.blender.org/D15617
Added logic to the USD Preview Surface importer to
convert UDIM textures.
Reviewed by: Sybren and Jesse
Differential Revision: https://developer.blender.org/D15379
This is a partial fix for T90535.
USD allows binding materials generically as well as for a
specific purpose. I.e., purpose may be generic (unspecified)
or one of
- Full: truest representation of the scene
- Preview: lightweight material for preview
Curently, only generically bound materials, with unspecified
purpose (allPurpose), are imported. This issue is preventing
preview materials from being imported in the Alab scene.
This patch adds logic to attempt to fall back on importing
preview or full materials, in that order, if there is no
generic material bound to the mesh.
Reviewed by: Sybren
Differential Revision: https://developer.blender.org/D15352
Previously, when creating "very large" (tens-hundreds of thousands)
amounts of objects, the Blender code that was ensuring name
uniqueness was the bottleneck. That got recently addressed (D14162),
however now sorting of IDs by their names is the remaining bottleneck.
Name sorting code in Blender is optimized for the pattern where names
are inserted in already sorted order (i.e. objects expect to get added
near the end of the list). By doing this pre-sorting of objects
intended to get created by an importer (USD and OBJ, in this patch),
this sorting bottleneck can be largely removed, especially with very
high object counts.
Windows, Ryzen 5950X, import times:
- OBJ, splash screen scene (26k objects): 22.0s -> 20.7s
- USD, Disney Moana scene (250k objects): 585s -> 82.2s (10 minutes -> 1.5 minutes)
Reviewed By: Michael Kowalski, Howard Trickey
Differential Revision: https://developer.blender.org/D15506
The importer parts that were doing assignment of materials to the
imported objects/meshes were essentially having a quadratic complexity
in terms of scene object count. For each material assigned to each
object, they were scanning the whole scene, checking which other
Objects use the same Mesh data, in order to resize their material
arrays to match the size.
Performance details (Windows, Ryzen 5950X):
- Import OBJ Blender 3.0 splash scene (24k objects): 43.0s -> 32.9s
- Import USD Disney Moana scene (260k objects): saves two hours
(~7400s). Note that later on this crashes when trying to render the
imported result; crashes in the same way/place both in master and
this patch.
Implementation details:
The importers were doing "scan the world" basically twice for each
object, for each material: once when creating a new material slot
(assigns an empty material), and then again when assigning the
material.
However, all these importers (USD, Alembic, OBJ) always create one
Object for one Mesh. So that whole quadratic complexity resulting
from "scan the world for possible other users of this obdata" is
completely not needed; it just never finds anything. So add a new
dedicated function BKE_object_material_assign_single_obdata that skips
the expensive part, but should only be used when the caller knows that
the obdata has exactly one user (the passed object).
Reviewed By: Bastien Montagne, Michael Kowalski
Differential Revision: https://developer.blender.org/D15145
Many existing importers/exporters do log the time it takes to system
console (some others log more information too). In particular, OBJ
(C++ & python), STL (C++ & python), PLY, glTF2 all log the time it
takes. However, neither USD nor Alembic do. And also it's harder to
know the time it takes there from a profiler, since all the work
normally is done on a background job and is split between several
threads (so you can't just find some top-level function and see how
much time it took).
This change:
- Adds import/export time logging to USD & Alembic importer/exporter,
- In the time utility class (also used by OBJ & STL), improve the
output formatting: 1) print only one decimal digit, 2) for long
times, print seconds and also produce a hours:minutes:seconds form.
Reviewed By: Michael Kowalski, Kévin Dietrich
Differential Revision: https://developer.blender.org/D15170
Previous code was rebuilding "name to material" map for each object
being imported. Which means O(N*M) complexity (N=object count,
M=material count). There was already a TODO comment suggesting that
a single map that's maintained for the whole import would be enough.
This commit does exactly that.
While importing Moana USD scene (260k objects, 18k materials) this
saves about 6 minutes of import time.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15222
Previous code was doing N collection syncs when importing N objects
(essentially quadratic complexity in terms of object count). New
code avoids all the intermediate syncs by using
BKE_layer_collection_resync_forbid and
BKE_layer_collection_resync_allow, and then does one
BKE_main_collection_sync + BKE_main_collection_sync_remap for the
whole operation. The things done on the importer objects that are
dependent on the sync happening (marking them selected) are done in a
separate loop after the sync.
Timings: importing Moana USD scene (480k objects) on Windows, VS2022
Release build, AMD Ryzen 5950X: 12344sec -> 10979sec (saves 22 minutes).
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15215
This is a partial fix for T90535.
Added Material Name Collision USD import menu option, to specify
the behavior when USD materials in different namespaces have the
same name.
The Material Name Collision menu options are
- Make Unique: Import each USD material as a unique Blender material.
- Reference Existing: If a material with the same name already
exists, reference that instead of importing.
Previously, the default behavior was to always keep the existing
material. This was causing an issue in the ALab scene, where
dozens of different USD materials all have the same name,
usdpreviewsurface1, so that only one instance of these materials
would be imported.
Reviewed by: Sybren
Differential Revision: https://developer.blender.org/D14869
