Commit ea97bb1641 introducing the GHash mapping between objects and
their CollectionObject items in a Collection broke ID remapping of
collections's objects. Release builds would 'work', but debug builds
would assert in several ways when opening complex production files.
The root of the issue was a bad/missing handling of the 'duplicate case'
(several CollectionObjects pointing to a same Object).
While fixing the code was possible, it turned out to require disabling
to much safety checks. Further more, there was an opportunity to improve
efficiency of the related code in ID remapping (the pos-processing
checking for NULL and duplicates in collections objects lists).
This commit introduces a new 'dirty' tag for CollectionObject lists and
their ghash mappings.
This tag is set by the `foreach_id` callback when the `ob` pointer of a
CollectionObject is changed, and it is detected as (potentially)
breaking the consistency of that data.
This tag is then used by a new `BKE_collections_object_remove_invalids`
call, to only check and fix collections tagged as dirty, instead of all
the collections in the given Main. It replaces the previous
`BKE_collections_object_remove_nulls` and
`BKE_collections_object_remove_duplicates` functions.
The speed-up is about an order of magnitude for the clean-up code
itself, which gives 2-3 percent speed-up on resynching a complex
production file e.g.
This commit also includes some cleanups and re-organization of related
code.
Add a hash for faster look-ups on collection->gobject,
This avoids a full list lookup for every object added via Python's
CollectionObject.link as well as linking via BKE_collection_object_add_*
functions.
While the speedup is non-linear, linking & unlinking 100k objects from
Python is about 50x faster. Although unlinking all objects in order
(a best-case for linked lists) is approximately the same speed.
Ref !104553.
The issue was that geometry nodes was not reevaluated after changing the
content of the collection. That resulted in the other object still having a
reference to the deleted object, which resulted in a crash when it tried to
access it.
Adding the `ID_RECALC_GEOMETRY` tag indicates that the content of the
collection has changed and will trigger geometry nodes setups that depend
on the collection to update.
When a change happens which invalidates view layers the syncing will be postponed until the first usage.
This will improve importing or adding many objects in a single operation/script.
`BKE_view_layer_need_resync_tag` is used to tag the view layer to be out of sync. Before accessing
`BKE_view_layer_active_base_get`, `BKE_view_layer_active_object_get`, `BKE_view_layer_active_collection`
or `BKE_view_layer_object_bases` the caller should call `BKE_view_layer_synced_ensure`.
Having two functions ensures that partial syncing could be added as smaller patches in the future. Tagging a
view layer out of sync could be replaced with a partial sync. Eventually the number of full resyncs could be
reduced. After all tagging has been replaced with partial syncs the ensure_sync could be phased out.
This patch has been added to discuss the details and consequences of the current approach. For clarity
the call to BKE_view_layer_ensure_sync is placed close to the getters.
In the future this could be placed in more strategical places to reduce the number of calls or improve
performance. Finding those strategical places isn't that clear. When multiple operations are grouped
in a single script you might want to always check for resync.
Some areas found that can be improved. This list isn't complete.
These areas aren't addressed by this patch as these changes would be hard to detect to the reviewer.
The idea is to add changes to these areas as a separate patch. It might be that the initial commit would reduce
performance compared to master, but will be fixed by the additional patches.
**Object duplication**
During object duplication the syncing is temporarily disabled. With this patch this isn't useful as when disabled
the view_layer is accessed to locate bases. This can be improved by first locating the source bases, then duplicate
and sync and locate the new bases. Will be solved in a separate patch for clarity reasons ({D15886}).
**Object add**
`BKE_object_add` not only adds a new object, but also selects and activates the new base. This requires the
view_layer to be resynced. Some callers reverse the selection and activation (See `get_new_constraint_target`).
We should make the selection and activation optional. This would make it possible to add multiple objects
without having to resync per object.
**Postpone Activate Base**
Setting the basact is done in many locations. They follow a rule as after an action find the base and set
the basact. Finding the base could require a resync. The idea is to store in the view_layer the object which
base will be set in the basact during the next sync, reducing the times resyncing needs to happen.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15885
Related to {D15885} that requires scene parameter
to be added in many places. To speed up the review process
the adding of the scene parameter was added in a separate
patch.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15930
From re-checking related code, it seems that we already always ensure
consistency of the `lib` pointer between embedded IDs and their owners.
This commit only adds some asserts in ambedded ID read code to
double-check this.
Many existing .blend files (including startup ones) seem to have invalid
embedded IDs (they are not properly tagged with `LIB_EMBEDDED_DATA`).
We cannot `do_version` this so just fix it on the fly when detecting the
issue. User then need to re-save these files.
We also need to update some release files (default factory startup is
OK, but e.g. the VSE template one is not).
Keeping the assert is important here, as such missing flag is a critical
data corruption that can potentially have many serious consequences
throughout the ID management code.
Add a dedicated `owner_id` pointer to ID types that can be embedded
(Collections and NodeTrees), and modify slightly come code to make
handling those more safe and consistent.
This implements first part of T69169.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D15838
In some cases, there is a chance code already knows who might be the
owner of the given ID, in which case it can be more efficient to check
it first (especially in cases like embedded node trees or scene
collections, where the only other way is to loop over all possible
owners currently).
Will be used in next commit in some Outliner fix.
- batch rename
- keyframe settings
- tool name in Tool properties header
- tool name in Tool properties Drag (fake) enum
- new file templates
- new preset
- new text datablock
- new collection datablock
- new geometry nodes (modifier and node group)
- new grease pencil data (layers and materials)
Ref. T43295
Reviewed By: mont29
Differential Revision: https://developer.blender.org/D15533
Crash happened because code could not find a valid base in current scene
after adding the object, added some checks for that.
Root of the issue was wrong assumptions in `BKE_object_add` logic, which
would pick the first valid ancestor collection in case initially
selected collection was not editable. In some case, this could pick a
collection not instanced in the current scene's view layer, leading to
not getting a valid base for the newly added object.
Addressed this by adding a new variant of `BKE_collection_object_add`,
`BKE_collection_viewlayer_object_add`, which ensures final collection is
in given viewlayer.
Existing code for the `Move` operator, and some `Collections` panel
operations (Object properties) was absolutely not override-safe, and
sometimes not even linked-data safe.
The first element of the iterator was not being tested against the flag.
So in some cases it would lead to more objects been made into
single-user than the active (or selected) ones.
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
`BKE_collection_object_add` ensures given object is added to an editable
collection, and not e.g. a linked or override one.
However, some processes like do_version manipulate collections also from
libraries, i.e. linked collections, in those cases we need a version of
the code that unconditionnally adds the given object to the given
colleciton.
Fix is similar to how CollectionObject with NULL object pointers are handled.
Using one of the 'free' pad bytes in Object_Runtime struct instead of a
gset (or other external way to detect object duplicates), as this is
several times faster.
NOTE: This makes remapping slightly slower again (adds 10 extra seconds
to file case in T94059).
General improvements of remapping time complexity, especially when
remapping a lot of IDs at once, is a separate topic currently
investigated in D13615.
- Added space below non doc-string comments to make it clear
these aren't comments for the symbols directly below them.
- Use doxy sections for some headers.
- Minor improvements to doc-strings.
Ref T92709
The main goal of this refactor is to make BPath module use `IDTypeInfo`,
and move each ID-specific part of the `foreach_path` looper into their
own IDTypeInfo struct, using a new `foreach_path` callback.
Additionally, following improvements/cleanups are included:
* Attempt to get better, more consistent namings.
** In particular, move from `path_visitor` to more standard `foreach_path`.
* Update and extend documentation.
** API doc was moved to header, according to recent discussions on this
topic.
* Remove `BKE_bpath_relocate_visitor` from API, this is specific
callback that belongs in `lib_id.c` user code.
NOTE: This commit is expected to be 100% non-behavioral-change. This
implies that several potential further changes were only noted as
comments (like using a more generic solution for
`lib_id_library_local_paths`, addressing inconsistencies like path of
packed libraries always being skipped, regardless of the
`BKE_BPATH_FOREACH_PATH_SKIP_PACKED` `eBPathForeachFlag` flag value,
etc.).
NOTE: basic unittests were added to master already in
rBdcc500e5a265093bc9cc.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D13381
Two issues addressed here:
I) `asset_type_info` is sub-data, not a callback. Therefore, move it
before the callbacks in the `IDTypeInfo` struct.
II) More important, initialize this new attribute in *ALL* `IDTypeInfo`
instances. No member of this struct should ever be left implicitely
uninitilazed, ever.
Aftermath of rBa84f1c02d251.
rB43bc494892c3 switched `BKE_libblock_relink_to_newid` to use new ID
remapping and libquery code.
However, that new code does protect by default against remapping an
objects's data pointer when that object is in Edit mode, since this is
not a behavior that generic BKE code can handle (due to required editing
data for most obdata types when in edit mode).
So specific code that does create new IDs and need remapping in Edit
mode has to pass specific exception flags to remaping code.
This commit adds those remapping flags to `BKE_libblock_relink_to_newid`
and add said exception flag to the remapping call from
`ED_object_add_duplicate` when the object is in edit mode.
Handling of RigidBody data in duplicate of scenes/collections was very
wrong. This commit:
- Add handling of duplication of RB collections when fully duplicating
a scene.
- Fix Object duplication trying to add duplicated RB objects to
matching RBW collections.
While the later behavior is desired when only duplicated objects, when
duplicating their collections and/or scenes it is actually very bad, as
it would add back new object duplicates to old (RBW) collections.
We need to separate the flag telling duplicate code to not handle
remapping to new IDs etc., from the one telling the code that we are
currently duplicating a 'root' ID (i.e. not a dependency of another
duplicated ID).
This whole duplicate code/logic is still fairly unsatisfying, think it
will need further refactor, or maybe even re-design, at some point...
This was not really useful, and added estra useless steps in case and ID
should not actually be written.
Further more, it prevented clearing the usercount on write, which can be
cause a false positive 'chanhged' detection in undo/redo case.
This makes the internal naming consistent with the public API. And also gives
us a visibility_flag rather than restrictflag that can be extended with more
flags.
