When importing an USD, the Blender object names do not necessarily match
Prim names:
```
#usda 1.0
def Xform "xform1"
{
def Mesh "MyObject" # Will be imported as "MyObject"
}
def Xform "xform2"
{
def Mesh "MyObject" # Will be imported as "MyObject.001"
}
def Xform "xform2"
{
def Mesh "MyObject" # Will be imported as "MyObject.002"
}
```
This makes it difficult in the [USD Import Hooks]
(https://docs.blender.org/api/current/bpy.types.USDHook.html) to link a
Blender object back to its source Prim for additional processing. A
typical use cases for games is to generate UVs, create and apply a
material on the fly when importing a collision shape that does not have
a visual representation (hence no materials) based on some Prim
attributes, but that the artist needs to differenciate in Blender's
viewport.
The Import context exposes a new method `get_prim_map()` that returns a
`dict` of `prim path` / `list of Blender ID`.
For example, given the following USD scene,
```
/
|--XformThenCube [def Xform]
| `--Cube [def Cube]
|--XformThenXformCube [def Xform]
| `--XformIntermediate [def Xform]
| `--Cube [def Mesh]
|--Cube [def Cube]
`--Material [def Material]
`--Principled_BSDF [def Shader]
```
the `get_prim_map()` method will return a map as:
```python
@static_method
def on_import(import_context):
pprint(import_context.get_prim_map())
```
```json
{
"/Cube": [bpy.data.objects["Cube.002"], bpy.data.meshes["Cube.002"]],
"/XformThenCube": [bpy.data.objects["XformThenCube"]],
"/XformThenCube/Cube": [bpy.data.objects["Cube"], bpy.data.meshes["Cube"]],
"/XformThenXformCube": [bpy.data.objects["XformThenXformCube"]],
"/XformThenXformCube/XformIntermediate": [bpy.data.objects["XformIntermediate"]],
"/XformThenXformCube/XformIntermediate/Cube": [bpy.data.objects["Cube.001"], bpy.data.meshes["Cube.001"]],
"/Material": [bpy.data.materials["Material"]],
}
```
Co-authored-by: Odréanne Breton <odreanne.breton@ubisoft.com>
Co-authored-by: Sttevan Carnali Joga <sttevan.carnali-joga@ubisoft.com>
Co-authored-by: Charles Flèche <charles.fleche@ubisoft.com>
Some changes to how argparse is used in render tests:
1. Use the common approach of one dash for single-letter options (`-b`)
and two dashes for longer options (`--blender`). In this commit that
just means changing single-dashed (`-testdir`) to double-dashed
(`--testdir`).
2. Remove unnecessary `nargs` arguments. The code was telling `argparse`
to put CLI arguments into a list of one item, and then had code to
turn that one-item list into the item itself. I've just removed the
`nargs` argument altogether, as that just produces the desired
value without requiring more code.
I've also removed `nargs="+"` from the handling of the `--blender`
parameter, as that allowed for multiple occurrences of `--blender
{path}` but was silently ignoring all of those except the first.
To ensure that required arguments are present, the code now uses
`required=True` instead of `nargs`.
3. Add a `description` parameter so that `--help` shows what the
test script actually does. Also it helps people (like me) who want
to figure out which blend file is actually being opened by the
test, without making the test itself more verbose.
No functional changes, except that you now cannot add multiple
`--blender` arguments any more (the CLI invocation will fail). This wasn't
used anywhere I could find, though.
Pull Request: https://projects.blender.org/blender/blender/pulls/131666
While looking into something else, I noticed that the OSL render tests
were not testing the Principled Volume tests.
This is because when running OSL tests, we skip all Principled BSDF
tests by ignoring all tests that start with `principled_`
(this was done due to noise differences between SVM and OSL).
However this had the knock on effect of skipping tests like
- `principled_absorption`
- `principled_blackbody`
- `principled_smoke`
in the volumetric test suite.
This commit fixes this issue by renaming all Principled BSDF tests to
`principled_bsdf_` and updating the block list to use this new name.
Ref: blender/blender-test-data!31
Pull Request: https://projects.blender.org/blender/blender/pulls/131540
Make use of our sparse value writer in more places. Namely, when using
animated Camera or Light properties, this will prevent a needless stream
of unchanging values being written into the USD file [1]. Skeletons and
armatures benefit too but less so as typically the primary benefit only
applies to the comparatively small `scale` transform attribute, which
typically remains unchanged from frame to frame.
The newly added `set_attribute` common code can, and eventually will, be
used to reduce boilerplate elsewhere where we do the same sparse writing
dance.
Ref #130759
Pull Request: https://projects.blender.org/blender/blender/pulls/131333
While adding tests for subd import I discovered that our vertex crease
data was imported incorrectly.
This PR adds tests and fixes:
- We tried to read crease sharpness data as ints instead of floats. This
caused the import process to early-return, meaning we never load any
data at all
- Empty USD data would still cause us to create the `crease_vert`
attribute unnecessarily
- Sharpness data needs clamped to 0-1 to handle USD's SHARPNESS_INFINITE
value of 10.0
- We need to fill the `crease_vert` span with 0's since incoming USD
data may not have values set for all the verts
Pull Request: https://projects.blender.org/blender/blender/pulls/131516
Rather than using a default "Anim" name when exporting armature
animations, use the animation's Action name instead. Likewise use the
UsdSkelAnimation name for the Action's name during import.
Blendshapes/shapekeys still use the default "Anim" name as there's no
action for these situations.
Pull Request: https://projects.blender.org/blender/blender/pulls/131021
This adds a `unit_test_compare` function to `Curves`.
Compares the curves. Curves are the same if:
* The number of points matches.
* The number of curves matches.
* The attribute names are the same and have the same type.
* The point attribute values are within the `threshold`.
* The curve topology matches (e.g. the curve sizes are the same).
* The curve attribute values are within the `threshold`.
* The indices of the points and curves are the same (can be ignored if this should be treated as the same).
The implementation reuses the same functions as the existing
comparison function for meshes.
Pull Request: https://projects.blender.org/blender/blender/pulls/131164
We've had a bunch of inconsistency between `channel_bag` and `channelbag` in the
code base. After discussion with @dr.sybren, we decided to standardize on
`channelbag` and also rename the camelcase `ChannelBag` to `Channelbag` to be
consistent with that.
This PR implements those changes.
Note that the reason we standardized on `channelbag` rather than `channel_bag`
is because it makes things clearer when stringing multiple terms together in
type and function names. For example, in `channelbag_fcurves_move()` it makes
it clear that `channelbag` describes one thing, rather than `channel` and `bag`
being two separate things.
No functional changes intended.
Pull Request: https://projects.blender.org/blender/blender/pulls/130988
`Action.slots.new()` in the Python API previously took either an ID or nothing
as a parameter. In the former case it would create a slot with the appropriate
`id_root` and name for that ID. In the latter case it would create a default
slot with an unspecified `id_root` and default name.
This had several issues:
1. You couldn't create a slot with a specific `id_root` without already having
an ID of that type. In theory this isn't a problem, but in practice in larger
scripts/addons you don't necessarily have such an ID on hand at the call
site.
2. You couldn't directly create a slot with a desired name without an existing
ID with that name. This isn't so important, since you can always just set the
name afterwards. But it's a bit annoying.
3. Most other `new()` APIs in Blender *require* you to specify the name of the
item being created. So calling this with no parameters was violating that
norm.
4. Ideally, we want to eliminate unspecified `id_root`s, since they cause other
weirdness in the API such as slot identifiers changing upon slot assignment.
To resolve these issues, and just generally to make the API more
straightforward, this PR changes `slots.new()` to take two required parameters:
an ID type and a name. For example:
`slots.new(id_type='CAMERA', name="My Camera Data Slot")`.
This fully specifies everything needed for the slot identifier upon creation,
and doesn't require any outside data items to create a slot with the desired
type and name.
In the future if we decide we still want a `for_id`-style slot creation API, we
can reintroduce it as a separate function.
Ref: #130892
Pull Request: https://projects.blender.org/blender/blender/pulls/130970
Adding object conversion tests to verify the refactored object type
conversion code works as intended.
The object conversion operator `exec()` function is restructured by
PR #130668 to clean up code path for better readability, since it's a
relatively big change, this test is put up to verify that:
- For supported object type pairs, converted objects would have the
expected target object type.
- For unsupported/unimplemented type pairs, the test expect the
converted object to have the same type as the source object.
- Verify that the code behaviour stays exactly the same as prior
to the change in #130668, including code logic that are later subject
to change and redesign, in which case this test file/script would need
to be updated accrodingly.
This test added such files:
- `tests/data/modelling/object_conversion.blend`: Sets up differnt types
of input objects and their expected converted object combinations.
- `tests/python/object_conversion.py`: Does mesh equal checks for
objects that are converted to mesh against expected objects in the
blend file, and does object type checks for other target types.
Note:
This test currently does not do topology-level equal check except for
when target object type is `MESH`. Equal check hasn't been implemented
for other object types yet, but ideally in the future we should use
topology check on top of type check for other object types as well.
See https://projects.blender.org/blender/blender/pulls/130668
Pull Request: https://projects.blender.org/blender/blender/pulls/130738
`AnimData`, NLA strips, and action constraints all have an `action_slot_name`
field in RNA. The purpose of this field is to store the identifier of the most
recently assigned slot, so that it can be used for auto-assignment when later
assigning different actions.
However, this field name is misleading in two ways:
1. In accordance with #130740, it's actually the slot *identifier*, not name.
2. It could be mistaken as a way to rename the currently assigned slot, which it
is not.
To resolve both of those issues, we're renaming the field to
`last_slot_identifier`, which better communicates its actual nature.
As a bonus, this also ends up decluttering Python autocomplete when looking
for things related to action_slot.
Ref: #130892
Pull Request: https://projects.blender.org/blender/blender/pulls/130911
This PR renames `ActionSlot::name` to `ActionSlot::identifier` for both DNA and
RNA, and also renames related methods, functions, constants, and comments.
The purpose of this rename is to help make it clear that this is not a "name"
in the traditional sense, but rather is a composite of the slot name + id type
for lookup purposes.
Ref: #130892
Pull Request: https://projects.blender.org/blender/blender/pulls/130740
Previously, it wasn't possible to copy a drawing from one frame to another
if that drawing was in another layer or object.
This allows the `frame.drawing` property to be set to any another drawing.
Setting `drawing` to `None` clears the drawing geometry.
Pull Request: https://projects.blender.org/blender/blender/pulls/130523
This patch makes it so collection assignments of bones are
symmetrized when the "Symmetrize" operator is run.
* Only collections with names that can be symmetrized are modified.
* Collections are created if they don't exist,
they are created under the same parent as the source collection
* In the case of a "left" bone assigned to a "right" collection the
resulting "right" bone will be assigned to a "left" collection.
The system does explicitly not try to be smart here.
Also adds unit tests.
Pull Request: https://projects.blender.org/blender/blender/pulls/130524
When EEVEE next removed the Alpha Threshold option a node graph setup
was used instead. Based on a USD test asset[1] the original code was
designed around only using the Image A channel during import to wire
up the nodes. However, the other channels (R, G, or B) are just as
valid.
We now handle this by wiring up the Separate RGB node, that is already
inserted for the scenario in general, to the shader nodes handling the
threshold calculation. Also adds additional test coverage for both
export and import as well.
The Round and LessThan+OneMinus networks remain the only shapes that
will be recognized for the processing.
[1] https://github.com/usd-wg/assets/tree/main/test_assets/AlphaBlendModeTest
Pull Request: https://projects.blender.org/blender/blender/pulls/130346
The existing point instancer reader is slightly refactored to allow for
animated setups. The primary change is simply to inherit from
`USDGeomReader` rather than `USDXformReader`. This allows access to the
`read_geometry` API used by the cache modifier.
The existing `read_object_data` method is split into two parts with
`read_geometry` loading per-frame USD data and `read_object_data`
coordinating the initial loading process, including creating the GN node
tree just once.
A new test has been added (a variation of the nested point instancer
file) with time samples on various attributes and on both point
instancers. This also fixes#129502 and the file provided in that issue.
----
The already added test file is `tests/data/usd/usd_point_instancer_anim.usda`
Pull Request: https://projects.blender.org/blender/blender/pulls/129881
Adds the option `merge_parent_xform` to the USD export operator and
panels so the transform and shapes are merged into a single USD Prim.
Without the option (existing default), a top-level mesh would be
exported as a top-level `Xform` that has a `Mesh` child:
```
def Xform "MyBlenderMeshObject"
{
matrix4d xformOp:transform = ...
def Mesh "MyBlenderMeshData"
{
}
}
```
This matches the Blender data model, where a transformable object
contains a geometric shape (like a mesh). This structure is also very
valid in USD, where we don't want to directly instantiate geometric
primitives[1]
However, "since number of prims on a stage is one of the primary factors
that governs how USD scale"[2], to reduce the number of prims in a
stage, geometric primitives *are transformable* themselves (see the
inheritence diagram[3]).
As such, the new export option allows to export geometric primitives
without the parent transform:
```
def Mesh "MyBlenderMeshObject"
{
matrix4d xformOp:transform = ...
}
```
This MR adds a the `is_object_data_context` flag to the
`HierarchyContext` context structure. The point of this change is to
make unambiguous in `USDHierarchyIterator::create_usd_export_context`
the fact that an `object` or a `data` is currently being exported: the
new `merge_parent_xform` option is meaningless for `object`. Only `data`
can be exported with a parent `Xform` or as a child of said `Xform`.
Ideally this flag would not be needed at all: the final USD prim path
*could* be computed in an override of the virtual
`AbstractHierarchyIterator::get_object_data_path` method. However, this
would mean that an `object` and a `data` would have the same export path.
This does not currently play well with
`AbstractHierarchyIterator::ensure_writer`, where `writers` are cached
*by their export path*: it would cache a transform writer, but will skip
the subsequent data writer.
Additionally, another new `is_parent` flag is added to handle the case
where merging the Xform is invalid: objects that are parents to other
objects should remain unmerged as otherwise that would yield invalid USD
files.
[1] https://openusd.org/release/glossary.html#usdglossary-gprim
[2] https://openusd.org/release/glossary.html#usdglossary-instancing
[3] https://openusd.org/release/api/class_usd_geom_xformable.html
Co-authored-by: Odréanne Breton <odreanne.breton@ubisoft.com>
Co-authored-by: Sttevan Carnali Joga <sttevan.carnali-joga@ubisoft.com>
Co-authored-by: Charles Flèche <charles.fleche@ubisoft.com>
Tell ffmpeg swscale to do accurate YUV->RGB conversion, instead of
slightly faster but not really accurate one. Fixes banding and some
color shifts in video files, particularly in dark regions.
The accurate conversion is a bit slower though, on 4K resolution video,
time taken to convert video frame from YUV to RGB:
- x64 (Ryzen 5950X): 2.3ms -> 3.7ms
- arm64 (M1 Max): 0.6ms -> 2.9ms
My take is that paying 1-2ms per 4K video playback is acceptable
since the result is obviously "more correct" and matches what VLC/ffplay
produces.
From what I can tell, "accurate conversion" turns off some dedicated
assembly code paths within ffmpeg. Maybe someday ffmpeg would get
accurate and assembly-optimized routines for that.
With more accurate decoding, we can now lower the expected render
test threshold again, since x64 & arm64 decoding is much closer now.
Comparison screenshots in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/130383
While adding test coverage for in-memory and packed texture scenarios, I
found that UDIMs were not being handled correctly in both cases. For
in-memory scenarios the per-tile generated/dirty status was not taken
into account. For packed scenarios the wrong filename substitutions were
being used.
This fixes both of these cases and adds test coverage for these
scenarios now. Both relative and absolute path options are validated.
Note: Both in-memory and packed images behave incorrectly when using the
'KEEP' and 'PRESERVE' texture export modes, so those remain untested
currently. A design on exactly what should happen in these modes is TBD.
Pull Request: https://projects.blender.org/blender/blender/pulls/130391
This new import option will allow users to chose if we merge USD prims
with their Xform parent.
Given a USD like:
```
def Xform "MyObject"
{
def Mesh "MyObject_LOD0"
{
}
}
```
When the option is set to True (existing default), only the mesh will be
imported and its parent Xform transformation will be baked into the
Mesh.
```
# In blender after import
- MyObject_LOD0 (Mesh)
```
When the option is set to False, the parent Xform will always be
imported as an Empty object:
```
# In blender after import
- MyObject (Empty)
├─ MyObject_LOD0 (Mesh)
```
Co-authored-by: Odréanne Breton <odreanne.breton@ubisoft.com>
Co-authored-by: Sttevan Carnali Joga <sttevan.carnali-joga@ubisoft.com>
When symmetrizing a bone (in edit mode) any constraint subtargets were name flipped,
if possible and they exist. This only worked if the target is the armature of the bone
being flipped.
This patch changes that so a subtarget flip is always attempted for targets that are an armature.
Pull Request: https://projects.blender.org/blender/blender/pulls/129169
Specifically coverage of various YUV formats (4:2:0 and 4:4:4),
some bit depths (most 8 bit, some 10 bit), various colorspace
settings (untagged, bt709), and YUV color ranges (regular/limited vs
full/pc).
build_test_movies.sh in there generates all the (tiny) movie files
out of input color_chart.png using command line ffmpeg.
This enables material displacement for UsdPreviewSurface import and
export. Scenarios are limited by what's supported by the preview surface
itself. Namely only Object Space displacement can be used (no vector
displacement)[1] and the Midlevel and Scale parameters are maintained by
adjusting the scale-bias on the image texture controlling the Height
(this means that Midlevel and Scale must be constants).
Hydra/MaterialX support is more complicated. First, there is a bug which
prevents scalar displacment from working correctly and that needs USD
2408+ for the fix[2]. Second, is that there's an open question about
which coordinate system to use for MaterialX's vector displacement maps.
Lastly, Hydra GL does not render displacement, making verification using
only Blender impossible[3]. As a result, this PR only makes MaterialX
"ready" for support, but stops short of actually connecting the final
piece of the node graph until more of the above can be sorted out.
Tests are added which cover:
- Variations of Midlevel and Scale values
- A constant Height setup
- Negative scenarios checking that only Object space is supported
and that midlevel and scale need to be constants
[1] https://openusd.org/release/spec_usdpreviewsurface.html
[2] https://github.com/PixarAnimationStudios/OpenUSD/issues/3325
[3] https://forum.aousd.org/t/materialx-displacement-hydra-storm/1098/2
Pull Request: https://projects.blender.org/blender/blender/pulls/128909
In case the process creashes, the prints about blendfiles being
processed could fail to be captured by the test framework.
And split these tests in 32 slices now, 8 was becomming way too slow to
complete for each test.
The existing Volume export, which already supports VDB file sequences
and static volumes created inside Blender, is now extended to handle
dynamically created and modified volumes. This allows scenarios where a
Volume Displace modifier is placed over-top an existing VDB sequence or
when Geometry Nodes is used to create animated volumes procedurally.
Detection of what counts as animation is simplistic and mimics what has
been used for Meshes. Essentially if there are any modifiers on the
volume we assume that the volume is "varying" in some way. This can lead
to situations where new volume files are written unnecessarily.
Volume import was also adjusted to correctly set the sequence "offset"
value. This is required to properly handle the case when a VDB sequence
begins animating at a different frame than what's implied by the file
name. For example, a VDB file sequence with file names containing 14-19
but the user wants to animate on frames 8-13 instead.
Tests are added which cover:
- Animated VDB file sequences
- Animated Mesh To Volume where the mesh has been animated
- Animated Volume Displacement where displacement settings are animated
- Animated Volumes created with a Geometry Nodes simulation
----
New test data has been checked in: `tests/data/usd/usd_volumes.blend` and files inside `tests/data/usd/volume-data/`
Pull Request: https://projects.blender.org/blender/blender/pulls/128907
This commit takes the 'Slotted Actions' out of the experimental phase.
As a result:
- All newly created Actions will be slotted Actions.
- Legacy Actions loaded from disk will be versioned to slotted Actions.
- The new Python API for slots, layers, strips, and channel bags is
available.
- The legacy Python API for accessing F-Curves and Action Groups is
still available, and will operate on the F-Curves/Groups for the first
slot only.
- Creating an Action by keying (via the UI, operators, or the
`rna_struct.keyframe_insert` function) will try and share Actions
between related data-blocks. See !126655 for more info about this.
- Assigning an Action to a data-block will auto-assign a suitable Action
Slot. The logic for this is described below. However, There are cases
where this does _not_ automatically assign a slot, and thus the Action
will effectively _not_ animate the data-block. Effort has been spent
to make Action selection work both reliably for Blender users as well
as keep the behaviour the same for Python scripts. Where these two
goals did not converge, reliability and understandability for users
was prioritised.
Auto-selection of the Action Slot upon assigning the Action works as
follows. The first rule to find a slot wins.
1. The data-block remembers the slot name that was last assigned. If the
newly assigned Action has a slot with that name, it is chosen.
2. If the Action has a slot with the same name as the data-block, it is
chosen.
3. If the Action has only one slot, and it has never been assigned to
anything, it is chosen.
4. If the Action is assigned to an NLA strip or an Action constraint,
and the Action has a single slot, and that slot has a suitable ID
type, it is chosen.
This last step is what I was referring to with "Where these two goals
did not converge, reliability and understandability for users was
prioritised." For regular Action assignments (like via the Action
selectors in the Properties editor) this rule doesn't apply, even though
with legacy Actions the final state ("it is animated by this Action")
differs from the final state with slotted Actions ("it has no slot so is
not animated"). This is done to support the following workflow:
- Create an Action by animating Cube.
- In order to animate Suzanne with that same Action, assign the Action
to Suzanne.
- Start keying Suzanne. This auto-creates and auto-assigns a new slot
for Suzanne.
If rule 4. above would apply in this case, the 2nd step would
automatically select the Cube slot for Suzanne as well, which would
immediately overwrite Suzanne's properties with the Cube animation.
Technically, this commit:
- removes the `WITH_ANIM_BAKLAVA` build flag,
- removes the `use_animation_baklava` experimental flag in preferences,
- updates the code to properly deal with the fact that empty Actions are
now always considered slotted/layered Actions (instead of that relying
on the user preference).
Note that 'slotted Actions' and 'layered Actions' are the exact same
thing, just focusing on different aspects (slot & layers) of the new
data model.
The "Baklava phase 1" assumptions are still asserted. This means that:
- an Action can have zero or one layer,
- that layer can have zero or one strip,
- that strip must be of type 'keyframe' and be infinite with zero
offset.
The code to handle legacy Actions is NOT removed in this commit. It will
be removed later. For now it's likely better to keep it around as
reference to the old behaviour in order to aid in some inevitable
bugfixing.
Ref: #120406
