DEG_add_collision_relations was only adding relationships for objects in
the brush collection, however dynamic paint also updates parent objects
and armature poses which weren't accounted for. Failing to include these
relations meant the evaluating the depsgraph could evaluate a pose
object from multiple threads at once - causing a crash.
Resolve by sharing logic between DEG_add_collision_relations &
BKE_object_modifier_update_subframe so depsgraph relationships match
objects the dynamic-paint modifier updates as part of its evaluation.
Ref !144844
The core issue here seems to be that the pasted object is turned into an Empty
but it still has all the modifiers. Later when the object is copied by the
depsgraph, the modifiers are not copied along, confusing some code.
Right now, this patch is rather defensive, so it solves the specific crash and
maybe similar cases that could come up in the future. One could separately also
clear all modifiers when turning the object into an Empty.
Pull Request: https://projects.blender.org/blender/blender/pulls/143029
Ultimately, the issue was caused by updating sound sequence handle in
`AUD_SequenceEntry_setSound` on each new frame regardless if the
modifier data has changed.
This commit fixes the issue by implementing a means for modifiers to
check, if their parameters or inputs are changed.
This is done by storing these parameters in `StripModifierDataRuntime`
struct, that is shared between all modifier types. This is not ideal,
but it significantly simplifies dependency graph runtime store/restore
code.
Function `strip_update_sound_modifiers` passes boolean `needs_update`
to strip stack update functions. If any needs to be updated, it sets
value of `needs_update` to true allowing following update functions to
skip parameter checking to speed up the process.
Original code updated sound sequence handle twice. Once by function
`BKE_sound_update_scene_sound` then by `strip_update_sound_modifiers`.
If sound modifier is used, only `strip_update_sound_modifiers` needs to
be called, so there is condition to decide which one of these functions
is called.
Also fixes#139605
Pull Request: https://projects.blender.org/blender/blender/pulls/141595
* Remove bke, ed and wm prefixes
* Add prefixes like: geom, object, blend, lib.
* Shorten some category names
* A few log level changes to improve --log-level info output
Pull Request: https://projects.blender.org/blender/blender/pulls/140244
The root cause of the crash was that currently the depsgraph does not support
being rebuilt twice without being evaluated in the meantime. While not ideal to
rebuild the depsgraph twice, it's really something I'd expect to work without
crashes/leaks.
The double-rebuild when switching the scene was introduced by b6e1afb6e1
which tagged the depsgraph relations indirectly. Tagging relations at that place
should be valid though. The same bug can easily be reproduced by explicitly
writing code that rebuilds the depsgraph twice as shown in
https://projects.blender.org/blender/blender/issues/139079#issuecomment-1615029.
So far, we've found two places that need to be fixed to properly support
rebuilding the depsgraph before it has been evaluated:
* `update_invalid_cow_pointers`: `previously_visible_components_mask` was used
to check if the id is new and therefore not expanded yet. However, this check
is wrong in the case when the depsgraph was not evaluated yet. Instead, check
whether the ID is expanded directly. IDs which don't use copy-on-eval are
still handled properly due to another existing check.
* `DepsgraphNodeBuilder::begin_build`: This just discarded
allocated-but-not-expanded IDs without freeing them. Now they are freed when
their ownership is not transferred to `IDInfo`.
See
https://projects.blender.org/blender/blender/issues/139079#issuecomment-1615029
for more details.
Pull Request: https://projects.blender.org/blender/blender/pulls/141199
This commit cleanly splits IDProperties storage for its two very different
usages:
* "User-defined" data, also known as "custom properties". Mostly exposed
in UI and as 'dictionary' in Python scripting, these are unique to each data
(each object can have its own set of custom properties).
* "System-defined" data, mainly as backend-storage for runtime RNA
structures and properties. While these are not necessarily present in the
storage, they are registered for a data type, and therefore always available
to all data of that type through RNA access.
See #123232 for rationales, designs and alternative investigated solutions.
## User-facing Changes
When using Blender, the only noticeable change is that python-defined RNA
data are not listed anymore in the Custom Properties panels (e.g. Cycles
data).
From a Python API perspective, the main changes are:
* Runtime RNA structs defined by sub-classing `PropertyGroup` and
registering them are no more accessible through the 'dict' syntax.
* They remain accessible through a dedicated 'bl_system_properties_get()`
callback, but its usages are only expected to be for testing and
debugging.
* The result of this call will be `None` by default when there has been
nothing written into it yet, unless its optional `do_create` parameter
is set to `True`.
* Some types (like `Node`, `UIList`, etc.) cannot store raw IDProperties
anymore (using the 'dict' syntax).
## Technical Details
* Adds System idprops to some data types (IDs, ViewLayer...).
* Moves some other containers (e.g operator properties, or some UI types like
UILists) to only have system-defined properties.
* For a few specific types (like `PropertyGroup`), the same storage is used,
but exposed in RNA as both user and system properties.
* Adds RNA API accessor callback to system idprops.
* Adds a function `bl_system_properties_get()`, which wraps system-defined
idprops and gives 'dict-like' access to them. Currently mainly used by some
unittests.
* System IDProps are always ignored by RNA diffing code (and therefore
liboverride processing), as their value is already exposed through RNA
properties, and should always be processed through these RNA properties.
* Modifies bpy rna binding to use these new system idprops for property
accesses, and keeps using user-defined idprops for 'dict-type' accesses.
* Handles versioning by copying 'user idprops' (existing ones) into new
'system idprops'.
### IDProperties Split
These types keep their historic IDProperty storage for user properties,
and get a new `system_id_properties` storage for system properties:
`ID`, `ViewLayers`, `Bone`, `EditBone`, `BoneCollection`, `PoseBone`, `Strip`
These types can both be extended with registrable RNA properties, and
expose Custom Properties in the UI.
### IDProperties become System Ones
These types keep a single IDProperties storage (their DNA struct does not
change), but it is now exclusively used for system-defined properties.
`OperatorProperty`, `View3DShading`, `UIList`, `AddonPreferences`,
`KeyConfigPreferences`, `GizmoProperties`, `GizmoGroupProperties`,
`Node`, `NodeSocket`, `NodeTreeInterfaceSocket`, `TimelineMarker`,
`AssetMetaData``
Since user properties were never available in the UI for them, they lose
their 'dict-like' IDProperties access in the Python API.
### Single Storage, Exposed as Both in API
These types still use a single IDProperty storage, but expose it both as
user properties and as system ones through RNA API.
* `PropertyGroup`: They need both access APIs since they are both
used for raw IDProperty groups (the 'dict-like' access), and
internally to access data of runtime-defined RNA structs.
* `IDPropertyWrapPtr`: Done mainly to follow `PropertyGroup`.
* `NodesModifier`: cannot become purely system idprops currently, as
there is no other access than the 'raw ID properties' paths to their
values. This can be changed once #132129 is finally implemented.
Pull Request: https://projects.blender.org/blender/blender/pulls/135807
When creating depsgraph relationships for drivers, avoid creating the
'unshare' depsgraph node for drivers on custom properties and on
shapekey `value` properties.
This should fix a significant part of the performance regression
mentioned in #140706.
Pull Request: https://projects.blender.org/blender/blender/pulls/140724
Replace Listbase with a VectorList for duplist generation.
Performing one heap allocation for each dupli becomes very costly on
scenes with many instances.
Using a VectorList can make playback up to 15% faster.
Most scenes will have lower performance improvements,
but none of the ones I tested were slower.
VectorLists also have the advantage of being able to query the size of
the duplilist.
Pull Request: https://projects.blender.org/blender/blender/pulls/138947
This is part of the short term roadmap goal of simplifying the
compositor workflow
(see https://projects.blender.org/blender/blender/issues/134214).
The problem is that many users don't know how to get started with
compositing in Blender, even when they have used Blender for other
areas, e.g. modeling.
Note: although the solution makes compositor node trees reusable
accross blend files, this is a nice side effect and not the main goal
of the PR.
This PR implements a "New" button that creates a new compositing node
tree, and manages trees as IDs. This has following advantages:
- Consistent with other node editors and other parts of Blender,
therefore making it easier to getting started with compositing if users
are familiar with shading or geometry nodes
- Give users the ability to reuse the compositing node tree by linking
or appending it.
Note: The parameter "Use Nodes" is still present in this PR, but will
be removed (in a backward compatible way) in a follow up PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/135223
Not only depend on the camera position but also on the other camera parameters.
This is important because otherwise Geometry Nodes won't be updated when e.g.
the focal length changes which is important when implementing e.g. camera
culling.
This already works when depending on a specific camera directly, but not when
depending on the active camera.
Pull Request: https://projects.blender.org/blender/blender/pulls/140046
Previously, the modifier name was used to identify it in a compute context or
viewer path. Using `ModifierData.persistent_uid` (which was only introduced
later) has two main benefits: * It is stable even when the modifier name
changes. * It's cheaper and easier to work with since it's just an integer
instead of a string.
Note: Pinned viewer nodes will need to be re-pinned after the change.
Pull Request: https://projects.blender.org/blender/blender/pulls/138864
Like other runtime structs, it doesn't make sense to write this
data to files. And moving it out of DNA to an allocated C++ struct
means we can use other C++ features in it, like the new Mutex
type which I switched to in this commit.
Pull Request: https://projects.blender.org/blender/blender/pulls/138551
This patch adds a new `BLI_mutex.hh` header which adds `blender::Mutex` as alias
for either `tbb::mutex` or `std::mutex` depending on whether TBB is enabled.
Description copied from the patch:
```
/**
* blender::Mutex should be used as the default mutex in Blender. It implements a subset of the API
* of std::mutex but has overall better guaranteed properties. It can be used with RAII helpers
* like std::lock_guard. However, it is not compatible with e.g. std::condition_variable. So one
* still has to use std::mutex for that case.
*
* The mutex provided by TBB has these properties:
* - It's as fast as a spin-lock in the non-contended case, i.e. when no other thread is trying to
* lock the mutex at the same time.
* - In the contended case, it spins a couple of times but then blocks to avoid draining system
* resources by spinning for a long time.
* - It's only 1 byte large, compared to e.g. 40 bytes when using the std::mutex of GCC. This makes
* it more feasible to have many smaller mutexes which can improve scalability of algorithms
* compared to using fewer larger mutexes. Also it just reduces "memory slop" across Blender.
* - It is *not* a fair mutex, i.e. it's not guaranteed that a thread will ever be able to lock the
* mutex when there are always more than one threads that try to lock it. In the majority of
* cases, using a fair mutex just causes extra overhead without any benefit. std::mutex is not
* guaranteed to be fair either.
*/
```
The performance benchmark suggests that the impact is negilible in almost
all cases. The only benchmarks that show interesting behavior are the once
testing foreach zones in Geometry Nodes. These tests are explicitly testing
overhead, which I still have to reduce over time. So it's not unexpected that
changing the mutex has an impact there. What's interesting is that on macos the
performance improves a lot while on linux it gets worse. Since that overhead
should eventually be removed almost entirely, I don't really consider that
blocking.
Links:
* Documentation of different mutex flavors in TBB:
https://www.intel.com/content/www/us/en/docs/onetbb/developer-guide-api-reference/2021-12/mutex-flavors.html
* Older implementation of a similar mutex by me:
https://archive.blender.org/developer/differential/0016/0016711/index.html
* Interesting read regarding how a mutex can be this small:
https://webkit.org/blog/6161/locking-in-webkit/
Pull Request: https://projects.blender.org/blender/blender/pulls/138370
Previously to handle unevaluated objects in line art "load all"
iteration, a `include_objects` variable is added in depsgraph
iteration settings, and this was only processed by object iterator
but not for any of the dupli objects. Now `make_duplis_collection`
will also handle `include_objects`
The only case where this border case can lead to crash is that a
line art grease pencil object is inside one of the dupli collection,
which isn't a valid use in the first place. But this fix makes it
more robust.
Pull Request: https://projects.blender.org/blender/blender/pulls/137323
Checking whether relations already existed was a bottleneck when building
collection -> object hierarchy relations. The duplicate checking would be
unnecessary since an object can only be in a collection once, but this function
to build relations can be called multiple times for the same collection.
To avoid the need to check for duplicates, only run the relation creation
once for each collection. The `BuilderMap` gives a simple way to do that.
Note that because a collection might be built separately as a non-layer
collection, there may still be duplicate hierarchy relations.
Building a Depsgraph for 60k simple objects became 1.86x faster, from 2.39
to 1.29 seconds.
Pull Request: https://projects.blender.org/blender/blender/pulls/137737
* Remove `DEG_get_evaluated_object` in favor of `DEG_get_evaluated`.
* Remove `DEG_is_original_object` in favor of `DEG_is_original`.
* Remove `DEG_is_evaluated_object` in favor of `DEG_is_evaluated`.
Pull Request: https://projects.blender.org/blender/blender/pulls/138317
Add a new node to the dependency graph, to act as a single entry point
before drivers are evaluated in parallel.
The node will take all the driven RNA properties, and write their
current value to the property again. This ensures that any implicitly
shared data is copied to ensure writability. Subsequent concurrent
writes by the driver evaluation will then be safe, as the
thread-unsafe part has already been performed.
Fixes: #132423
Pull Request: https://projects.blender.org/blender/blender/pulls/135802
Ref: #132179
Renames:
- `Editing.act_seq` -> `Editing.act_strip`
- `SequenceModifierData` -> `StripModifierData`
- Its member `mask_sequence` is now `mask_strip`.
- `MetaStack.parseq` -> `MetaStack.parent_strip`
- Remaining function names/parameters that were not dealt with in #132748
- Various references to `seq` or `sequence` throughout code and docs when
referring to a strip
Also moves `_get` to the end of the renamed function names where
applicable for standardization (unless "by" or "from" are used).
There should be no changes to current behavior.
Pull Request: https://projects.blender.org/blender/blender/pulls/138077
Allows to log human-readable list of recalc bits that are set.
The usage is pretty simple:
```
printf("%s: %s %s\n", __func__, id->name, DEG_stringify_recalc_flags(id->recalc).c_str());
```
Pull Request: https://projects.blender.org/blender/blender/pulls/137715
The goal here is to avoid having to cast to and from `ID` when getting the
evaluated or original ID using the depsgraph API, which is often verbose and not
type safe. To solve this, there are now `DEG_get_original` and
`DEG_get_evaluated` methods which are templated on the type and use a new
`is_ID_v` static type check to make sure it's only used with valid types.
This allows removing quite some verbosity on all the call sites. I also removed
`DEG_get_original_object`, because that does not have to be a special case
anymore.
Pull Request: https://projects.blender.org/blender/blender/pulls/137629
Use `blender::Stack` instead of `BLI_Stack`. Theoretically this can
improve performance because it makes use of inlining to remove function
call overhead and give the compiler more information. I observed a build
time improvement of about 4%, from 1156 to 1113 ms with 30k cube objects.
Notably, now `deg_graph_flush_visibility_flags` doesn't call any
non-inlined functions.
Pull Request: https://projects.blender.org/blender/blender/pulls/137355
I observed allocation becoming a bottleneck when building the depsgraph
with scenes with many simple data-blocks. One of the main culprits was
the struct that encodes relations between nodes in the graph.
Instead of allocating each `Relation` with a separate allocation call,
combine them into a `LinearAllocator`. That is must faster because it
allocates in large chunks and just bumps an offset on each allocation.
In a test file with 30 thousand cube objects, I observe a 1.18x
improvement in depsgraph evaluation time, from 1370 to 1163 ms.
The depsgraph isn't completely re-allocated when it's rebuilt, so the
allocator added in this PR has to be cleared manually. In the future,
it can be used for other structs or potentially strings.
Pull Request: https://projects.blender.org/blender/blender/pulls/137303
Adding "writeback" callbacks to the depsgraph should be temporary, since these
callbacks can bind pointers that become invalid. In this case: the evaluated
nodes modifier bound by a bake node callback. These callbacks are cleared after
being added in the `deg_flush_updates_and_refresh` function, but there are other
cases where depsgraph updates are executed which don't support writeback
callbacks (`object_force_modifier_update_for_bind` run by the smooth modifier
"bind" function).
To prevent dangling invalid pointers in outdated callbacks, disallow adding
callbacks in any case other than the `deg_flush_updates_and_refresh` function.
Since callbacks can be added from any depsgraph update, the safest way to
prevent adding them is in the depsgraph itself. If the `use_writeback_callbacks`
flag is not set, any callback is simply discarded.
Pull Request: https://projects.blender.org/blender/blender/pulls/137083
When the collection is disabled and was only referenced by a node tree, it was
missing from the depsgraph. The Geometry Nodes modifier tried to add a relation
to it, but it couldn't because the collection was not in the depsgraph in the first place.
Now, the collection is always added to the depsgraph when it's referenced by a
node tree that's in the depsgraph too. This makes the existing code that adds the
relation from the collection to the Geometry Nodes modifier work.
Pull Request: https://projects.blender.org/blender/blender/pulls/136328
This implements bundles and closures which are described in more detail in this
blog post: https://code.blender.org/2024/11/geometry-nodes-workshop-october-2024/
tl;dr:
* Bundles are containers that allow storing multiple socket values in a single
value. Each value in the bundle is identified by a name. Bundles can be
nested.
* Closures are functions that are created with the Closure Zone and can be
evaluated with the Evaluate Closure node.
To use the patch, the `Bundle and Closure Nodes` experimental feature has to be
enabled. This is necessary, because these features are not fully done yet and
still need iterations to improve the workflow before they can be officially
released. These iterations are easier to do in `main` than in a separate branch
though. That's because this patch is quite large and somewhat prone to merge
conflicts. Also other work we want to do, depends on this.
This adds the following new nodes:
* Combine Bundle: can pack multiple values into one.
* Separate Bundle: extracts values from a bundle.
* Closure Zone: outputs a closure zone for use in the `Evaluate Closure` node.
* Evaluate Closure: evaluates the passed in closure.
Things that will be added soon after this lands:
* Fields in bundles and closures. The way this is done changes with #134811, so
I rather implement this once both are in `main`.
* UI features for keeping sockets in sync (right now there are warnings only).
One bigger issue is the limited support for lazyness. For example, all inputs of
a Combine Bundle node will be evaluated, even if they are not all needed. The
same is true for all captured values of a closure. This is a deeper limitation
that needs to be resolved at some point. This will likely be done after an
initial version of this patch is done.
Pull Request: https://projects.blender.org/blender/blender/pulls/128340
While profiling a scene with many objects, I noticed some unexpected
functions taking a significant time of depsgraph creation/evaluation.
String length calculation and equality comparison was taking longer than
it should, and some simple methods were appearing in profiles that should
be inlined instead.
There are more places where this sort of change would be helpful, this
commit just changes places using `OperationIDKey` and `ComponentIDKey`.
Pull Request: https://projects.blender.org/blender/blender/pulls/136795
This replaces the deprecated DrawData mechanism by the
usage of the update timestamp `last_update`.
The compositor keeps the `last_update` value of the cached ID
and compares it with the value on the ID at the time of evaluation.
Rel #134690
Pull Request: https://projects.blender.org/blender/blender/pulls/134878
This was cause by the legacy undo recreating the depsgraph for
every update.
Depsgraph is recreated for each undo step which make the update
counter stay at 1. The fix is to protect for this case using a
global atomic counter.
