Fixes issues in importers written in C++ (T100737):
- Materials had one reference count too much. Affected Collada,
Alembic, USD, OBJ importers, looks like "since forever".
- Active material index was not properly set on imported meshes.
Regression since 3.3 (D15145). Affected Alembic, USD, OBJ. Note:
now it sets the first material as the active one, whereas
previously the last one was set as active. First one sounds more
"intuitive" to me.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15831
This patch moves material indices from the mesh `MPoly` struct to a
generic integer attribute. The builtin material index was already
exposed in geometry nodes, but this makes it a "proper" attribute
accessible with Python and visible in the "Attributes" panel.
The goals of the refactor are code simplification and memory and
performance improvements, mainly because the attribute doesn't have
to be stored and processed if there are no materials. However, until
4.0, material indices will still be read and written in the old
format, meaning there may be a temporary increase in memory usage.
Further notes:
* Completely removing the `MPoly.mat_nr` after 4.0 may require
changes to DNA or introducing a new `MPoly` type.
* Geometry nodes regression tests didn't look at material indices,
so the change reveals a bug in the realize instances node that I fixed.
* Access to material indices from the RNA `MeshPolygon` type is slower
with this patch. The `material_index` attribute can be used instead.
* Cycles is changed to read from the attribute instead.
* BMesh isn't changed in this patch. Theoretically it could be though,
to save 2 bytes per face when less than two materials are used.
* Eventually we could use a 16 bit integer attribute type instead.
Ref T95967
Differential Revision: https://developer.blender.org/D15675
When allocating new `CustomData` layers, often we do redundant
initialization of arrays. For example, it's common that values are
allocated, set to their default value, and then set to some other
value. This is wasteful, and it negates the benefits of optimizations
to the allocator like D15082. There are two reasons for this. The
first is array-of-structs storage that makes it annoying to initialize
values manually, and the second is confusing options in the Custom Data
API. This patch addresses the latter.
The `CustomData` "alloc type" options are rearranged. Now, besides
the options that use existing layers, there are two remaining:
* `CD_SET_DEFAULT` sets the default value.
* Usually zeroes, but for colors this is white (how it was before).
* Should be used when you add the layer but don't set all values.
* `CD_CONSTRUCT` refers to the "default construct" C++ term.
* Only necessary or defined for non-trivial types like vertex groups.
* Doesn't do anything for trivial types like `int` or `float3`.
* Should be used every other time, when all values will be set.
The attribute API's `AttributeInit` types are updated as well.
To update code, replace `CD_CALLOC` with `CD_SET_DEFAULT` and
`CD_DEFAULT` with `CD_CONSTRUCT`. This doesn't cause any functional
changes yet. Follow-up commits will change to avoid initializing
new layers where the correctness is clear.
Differential Revision: https://developer.blender.org/D15617
Added logic to the USD Preview Surface importer to
convert UDIM textures.
Reviewed by: Sybren and Jesse
Differential Revision: https://developer.blender.org/D15379
This is a partial fix for T90535.
USD allows binding materials generically as well as for a
specific purpose. I.e., purpose may be generic (unspecified)
or one of
- Full: truest representation of the scene
- Preview: lightweight material for preview
Curently, only generically bound materials, with unspecified
purpose (allPurpose), are imported. This issue is preventing
preview materials from being imported in the Alab scene.
This patch adds logic to attempt to fall back on importing
preview or full materials, in that order, if there is no
generic material bound to the mesh.
Reviewed by: Sybren
Differential Revision: https://developer.blender.org/D15352
Previously, when creating "very large" (tens-hundreds of thousands)
amounts of objects, the Blender code that was ensuring name
uniqueness was the bottleneck. That got recently addressed (D14162),
however now sorting of IDs by their names is the remaining bottleneck.
Name sorting code in Blender is optimized for the pattern where names
are inserted in already sorted order (i.e. objects expect to get added
near the end of the list). By doing this pre-sorting of objects
intended to get created by an importer (USD and OBJ, in this patch),
this sorting bottleneck can be largely removed, especially with very
high object counts.
Windows, Ryzen 5950X, import times:
- OBJ, splash screen scene (26k objects): 22.0s -> 20.7s
- USD, Disney Moana scene (250k objects): 585s -> 82.2s (10 minutes -> 1.5 minutes)
Reviewed By: Michael Kowalski, Howard Trickey
Differential Revision: https://developer.blender.org/D15506
The importer parts that were doing assignment of materials to the
imported objects/meshes were essentially having a quadratic complexity
in terms of scene object count. For each material assigned to each
object, they were scanning the whole scene, checking which other
Objects use the same Mesh data, in order to resize their material
arrays to match the size.
Performance details (Windows, Ryzen 5950X):
- Import OBJ Blender 3.0 splash scene (24k objects): 43.0s -> 32.9s
- Import USD Disney Moana scene (260k objects): saves two hours
(~7400s). Note that later on this crashes when trying to render the
imported result; crashes in the same way/place both in master and
this patch.
Implementation details:
The importers were doing "scan the world" basically twice for each
object, for each material: once when creating a new material slot
(assigns an empty material), and then again when assigning the
material.
However, all these importers (USD, Alembic, OBJ) always create one
Object for one Mesh. So that whole quadratic complexity resulting
from "scan the world for possible other users of this obdata" is
completely not needed; it just never finds anything. So add a new
dedicated function BKE_object_material_assign_single_obdata that skips
the expensive part, but should only be used when the caller knows that
the obdata has exactly one user (the passed object).
Reviewed By: Bastien Montagne, Michael Kowalski
Differential Revision: https://developer.blender.org/D15145
Many existing importers/exporters do log the time it takes to system
console (some others log more information too). In particular, OBJ
(C++ & python), STL (C++ & python), PLY, glTF2 all log the time it
takes. However, neither USD nor Alembic do. And also it's harder to
know the time it takes there from a profiler, since all the work
normally is done on a background job and is split between several
threads (so you can't just find some top-level function and see how
much time it took).
This change:
- Adds import/export time logging to USD & Alembic importer/exporter,
- In the time utility class (also used by OBJ & STL), improve the
output formatting: 1) print only one decimal digit, 2) for long
times, print seconds and also produce a hours:minutes:seconds form.
Reviewed By: Michael Kowalski, Kévin Dietrich
Differential Revision: https://developer.blender.org/D15170
Previous code was rebuilding "name to material" map for each object
being imported. Which means O(N*M) complexity (N=object count,
M=material count). There was already a TODO comment suggesting that
a single map that's maintained for the whole import would be enough.
This commit does exactly that.
While importing Moana USD scene (260k objects, 18k materials) this
saves about 6 minutes of import time.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15222
Previous code was doing N collection syncs when importing N objects
(essentially quadratic complexity in terms of object count). New
code avoids all the intermediate syncs by using
BKE_layer_collection_resync_forbid and
BKE_layer_collection_resync_allow, and then does one
BKE_main_collection_sync + BKE_main_collection_sync_remap for the
whole operation. The things done on the importer objects that are
dependent on the sync happening (marking them selected) are done in a
separate loop after the sync.
Timings: importing Moana USD scene (480k objects) on Windows, VS2022
Release build, AMD Ryzen 5950X: 12344sec -> 10979sec (saves 22 minutes).
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15215
This is a partial fix for T90535.
Added Material Name Collision USD import menu option, to specify
the behavior when USD materials in different namespaces have the
same name.
The Material Name Collision menu options are
- Make Unique: Import each USD material as a unique Blender material.
- Reference Existing: If a material with the same name already
exists, reference that instead of importing.
Previously, the default behavior was to always keep the existing
material. This was causing an issue in the ALab scene, where
dozens of different USD materials all have the same name,
usdpreviewsurface1, so that only one instance of these materials
would be imported.
Reviewed by: Sybren
Differential Revision: https://developer.blender.org/D14869
No longer happens on the buildbot, but for users building with an older
Xcode, still need to avoid using value().
Differential Revision: https://developer.blender.org/D14883
Add support for volume (OpenVDB) USD export:
- Allows to export both static and animated volumes.
- Supports volumes that have OpenVDB data from files or are generated in
Blender with 'Mesh to Volume' modifier.
- For volumes that have generated data in Blender it also exports
corresponding .vdb files. Those files are saved in a new folder named
"volumes".
- Slightly changes the USD export UI panel. "Relative Texture Paths"
becomes "Relative Paths" (and has separate UI box) as the
functionality will now apply to both textures and volumes. Disabling
of this option due to "Materials" checkbox being turned off has been
removed.
Reviewed By: sybren, makowalski
Differential Revision: https://developer.blender.org/D14193
Manifest Task: T95407
The "PROP" in the name reflects its generic status, and removing
"LOOP" makes sense because it is no longer associated with just
mesh face corners. In general the goal is to remove extra semantic
meaning from the custom data types.
This adds a basic unit test to check USD has been correctly
build with imaging components to support building both with
the old and new libs, it automatically adds the test when it
detects a library with imaging enabled. (platform devs will
have to pay attention it runs the test to validate the libs
build correctly)
For future use in the code it also defines a USD_HAS_IMAGING
define one could check if we're building against an USD lib
that has it (just because we build/ship with it, doesn't
mean downstream builds will ship with it, so we'll have
to be a little pro-active there)
Reviewed By: sybren
Differential Revision:https://developer.blender.org/D14456
- Missing star prefix.
- Unnecessary indentation.
- Blank line after dot-points
(otherwise doxygen merges with the previous dot-point).
- Use back-slash for doxygen commands.
- Correct spelling.
Both the Alembic and USD libraries use double precision floating
point numbers internally to store time. However the Alembic I/O
code defaulted to floats even though Blender's Scene FPS, which is
generally used for look ups, is stored using a double type. Such
downcasts could lead to imprecise lookups, and would cause
compilation warnings (at least on MSVC).
This modifies the Alembic exporter and importer to make use of
doubles for the current scene time, and only downcasting to float
at the very last steps (e.g. for vertex interpolation). For the
importer, doubles are also used for computing interpolation weights,
as it is based on a time offset.
Although the USD code already used doubles internally, floats were used
at the C API level. Those were replaced as well.
Differential Revision: https://developer.blender.org/D13855
USD requires to be linked with /WHOLEARCHIVE so
the linker won't remove their static initializers.
This strangely has never worked for MSVC since
the flags were set on the LINK_FLAGS property
which is only used to link .dll and .exe files,
given this is a static lib, the flags were not
used, nor did CMake propagate the link directive
to the final targets that did link. Not quite sure
how this has not lead to more problems in the past.
Setting the link directive on the INTERFACE_LINK_OPTIONS
makes cmake do the right thing.
Differential Revision: https://developer.blender.org/D14394
Reviewed by: sybren
For 3.2 USD will be bumped to a newer version with some
slight API changes, however since we cannot simultaneously
land the libs for all platforms as well as these code changes,
we'll have to support both 21.02 and 21.11+ for at least a
short period of time making the code slightly more messy than
it could have been.
Differential Revision: https://developer.blender.org/D14184
Reviewed by: sybren
Fix provided by Piotr Makal in patch D14204.
This patch fixes volume grid duplication which was occurring during
importing USD files. This was caused by calling BKE_volume_grid_add
twice per grid (excluding 'density' grid) for the same Volume
object: (1) in USDVolumeReader::read_object_data and (2) later in
BKE_volume_load.
Differential Revision: https://developer.blender.org/D14204
This adds a const qualifier to some code path in the Alembic and USD
importers. More could be added elsewhere. This change is done as it will
be required when GeometrySets are supported and helps keeping diff noise
in the patch to a bare minimum.
Also fixes missing code to read/write/free/copy color management settings
in various places. This can't be set through the UI currently, but still
should be handled consistently.
Added call to ensure that the USD plugins are registered
when opening a USD cache archive. This is to avoid USD
load errors due to missing USD file format plugins when
opening blender files that contain USD transform cache
constraints and mesh sequence cache modifilers.
Fixes T94396
This commit renames enums related the "Curve" object type and ID type
to add `_LEGACY` to the end. The idea is to make our aspirations clearer
in the code and to avoid ambiguities between `CURVE` and `CURVES`.
Ref T95355
To summarize for the record, the plans are:
- In the short/medium term, replace the `Curve` object data type with
`Curves`
- In the longer term (no immediate plans), use a proper data block for
3D text and surfaces.
Differential Revision: https://developer.blender.org/D14114
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Changes to recent addition: c85c52f2ce.
Having both BLI_paths_equal and BLI_path_cmp made it ambiguous
which should be used, as `BLI_paths_equal` wasn't the equivalent to
`BLI_path_cmp(..) == 0` as it is for string equals macro `STREQ(..)`.
It's also a more specialized function which is not used for path
comparison throughout Blender's internal path handling logic.
Instead rename this `BLI_path_cmp_normalized` and return the result of
`BLI_path_cmp` to make it clear paths are modified before comparison.
Also add comments about the conventions for Blender's path comparison
as well as a possible equivalent to Python's `os.path.samefile`
for checking if two paths point to the same location on the file-system.
Add `USD Preview Surface From Nodes` export option, to convert a
Principled BSDF material node network to an approximate USD Preview
Surface shader representation. If this option is disabled, the original
material export behavior is maintained, where viewport setting are saved
to the Preview Surface shader.
Also added the following options for texture export.
- `Export Textures`: If converting Preview Surface, export textures
referenced by shader nodes to a 'textures' directory which is a
sibling of the USD file.
- `Overwrite Textures`: Allow overwriting existing texture files when
exporting textures (this option is off by default).
- `Relative Texture Paths`: Make texture asset paths relative to the
USD.
The entry point for the new functionality is
`create_usd_preview_surface_material()`, called from
`USDAbstractWriter::ensure_usd_material()`. The material conversion
currently handles a small subset of Blender shading nodes,
`BSDF_DIFFUSE`, `BSDF_PRINCIPLED`, `TEX_IMAGE` and `UVMAP`.
Texture export is handled by copying texture files from their original
location to a `textures` folder in the same directory as the USD.
In-memory and packed textures are saved directly to the textures folder.
This patch is based, in part, on code in Tangent Animation's USD
exporter branch.
Reviewed By: sybren, HooglyBoogly
Differential Revision: https://developer.blender.org/D13647
