Check if either the memory is zero or already matches the default value,
and copy. This simplifies a common pattern to a single line.
Preparing for default initializers in DNA (#134531).
Pull Request: https://projects.blender.org/blender/blender/pulls/138830
Remove the addition of the position attribute from the default
"init data" callback where we don't know the desired number
of points. Add it in the other functions that add the data-block
(except the version that purposefully doesn't add attributes).
Pull Request: https://projects.blender.org/blender/blender/pulls/138697
As described in #122398, implement read and write support for a new
attribute storage system. Currently this is only implemented to support
forward compatibility; the format used at runtime isn't changed at all.
That can be done one step at a time during the 4.5 and 5.0 development
cycles. A new experimental option for testing tells Blender to always
save with the new format.
The main benefit of the new structure is that it matches the attribute
system design, it allows for future attribute storage optimization, and
each attribute is an allocated struct, which will give pointer stability
for the Python API.
The next step is to connect the attribute API and the RNA API to
AttributeStorage for the simplest geometry type, point clouds.
Pull Request: https://projects.blender.org/blender/blender/pulls/133874
This adds a version of `BKE_id_new_nomain` that takes the ID type parameter as
template argument. This allows the function the return the newly created ID with
the correct type, removing the need to use `static_cast` on the call-site.
To make this work, I added a static `id_type` member to every ID struct. This
can also be used to create a similar API for other id management functions in
future patches.
```cpp
// Old
Mesh *mesh = static_cast<Mesh *>(BKE_id_new_nomain(ID_ME, "Mesh"));
// New
Mesh *mesh = BKE_id_new_nomain<Mesh>("Mesh");
```
Pull Request: https://projects.blender.org/blender/blender/pulls/138383
These were handled mostly completely outside of IDManagement code, yet
(ab)using the same ID management system in some cases, adding hacks to
address some issues, etc.
Also address a similar issue in the eevee lookdev `LookdevWorld` code.
Since initializing pre-allocated (or static) buffers as valid IDs is not
currently supported, and these use-cases do not seem common enough to be
worth supporting it currently, instead switch to storing allocated IDs
into static pointers.
This allows to use proper 'out-of-main' ID creation code API.
NOTE: There are still some remaining issues, especially in the GP
material BKE api. These are noted in comments for now, as it would be
out of scope to address them in this commit.
Pull Request: https://projects.blender.org/blender/blender/pulls/138263
Due to !133498 layers containing incorrect material indices could lead to generating
to many GPU resources crashing Blender. As there is a maximum number of material slots
we can limit it.
This is just a safeguard for incorrect values in material_index attribute layer. Working with
this material index attributes should automatically limit, but I leave that to the modeling
module.
Pull Request: https://projects.blender.org/blender/blender/pulls/136302
The main issue of 'type-less' standard C allocations is that there is no check on
allocated type possible.
This is a serious source of annoyance (and crashes) when making some
low-level structs non-trivial, as tracking down all usages of these
structs in higher-level other structs and their allocation is... really
painful.
MEM_[cm]allocN<T> templates on the other hand do check that the
given type is trivial, at build time (static assert), which makes such issue...
trivial to catch.
NOTE: New code should strive to use MEM_new (i.e. allocation and
construction) as much as possible, even for trivial PoD types.
Pull Request: https://projects.blender.org/blender/blender/pulls/136134
Unlike the legacy type, the radius isn't included in the bounds for the new
curves type. This hasn't been obvious because the drawing is quite broken
and doesn't use the radius properly.
This commit adds a separate cache for the bounds with the radius, which
is now used by default. The old cache is kept around for backward
compatibility in the bounding box geometry node, where a new
"Use Radius" option accesses the old behavior.
Pull Request: https://projects.blender.org/blender/blender/pulls/135584
Note: The point cloud bounding box is not being updated once the points are deleted.
This is a known issue with BKE_pointcloud_nomain_to_pointcloud and Hans is looking into it.
Code inspired/built based on the Delete Geometry node.
Pull Request: https://projects.blender.org/blender/blender/pulls/134622
Previously we generally expected CustomData layers to have implicit
sharing info, but we didn't require it. This PR clarifies that we do
require layers with non-null data to have implicit sharing info. This
generally makes code simpler because we don't have to have a separate
code path for non-shared layers. For example, it makes the "totelem"
arguments for layer freeing functions unnecessary, since shared data
knows how to free itself. Those arguments are removed in this PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/134578
Replace `const char *` with `StringRef` for the API in `BKE_attribute.h`.
The benefits are slightly simpler code and possibly slightly improved
performance through avoiding the need to measure string length.
Pull Request: https://projects.blender.org/blender/blender/pulls/134183
The core issue was that the geometry batch cache (e.g. `MeshBatchCache` or
`PointCloudBatchCache`) was dependent on the object. This is problematic when
the the same geometry is used with multiple different objects because the cache
can't be consistent with all of them.
Fortunately, the only thing that was retrieved from the object was the number of
material slots, so if that can be avoided we should be fine. We can't just use
the number of material slots stored on the geometry because that may have no
material slots but still has material indices which are overridden on the object
level.
The solution is to take make the number of materials for a geometry only
dependent on the actual `material_index` attribute and not on the number of
available slots. More specifically, we find the maximal referenced material
index and handle that many materials. This number does not depend on how many
material slots there are on the object, but it still allows the object to
override materials slots that the mesh references.
A downside is that the maximum material index has to be computed which often
requires an iteration over the mesh. Fortunately, we can cache that quite easily
and the computation can be done in parallel. Also we are probably able to
eagerly update the material index in many cases when it's set instead of
computing it lazily. That is not implemented in this patch though.
The largest part of the patch is making the maximal material index easily
available on all the geometry types. Besides that, the material API is slightly
replaced and the drawing code now makes use of the updated API.
Pull Request: https://projects.blender.org/blender/blender/pulls/133498
Previously, the attribute accessor were defined in the `geometry_component_*.cc`
files. This made sense back in the day, because this attribute API was only used
through `GeometryComponent`. However, nowadays this attribute API is independent
of `GeometryComponent`. E.g. one can use `mesh.attributes()` without ever
creating a component.
The refactor contains the following changes:
* Move attribute accessors to separate files for each geometry type. E.g. from
`geometry_component_mesh.cc` to `mesh_attributes.cc`.
* Move implementations of e.g. `Mesh::attributes()` to `mesh.cc`.
* Provide access to the `AttributeAccessorFunctions` without actually having a
geometry. This will be useful to e.g. implement
`attribute_is_builtin_on_component_type` without dummy components.
Pull Request: https://projects.blender.org/blender/blender/pulls/130516
And same for the `copy_layout` function. These functions do not free any
potentially existing data in destination, but expect it to be uninitialized.
Hopefully these new names will make it more clear and avoid bugs like #122160.
We often have the situation where it would be good if we could easily estimate
the memory usage of some value (e.g. a mesh, or volume). Examples of where we
ran into this in the past:
* Undo step size.
* Caching of volume grids.
* Caching of loaded geometries for import geometry nodes.
Generally, most caching systems would benefit from the ability to know how much
memory they currently use to make better decisions about which data to free and
when. The goal of this patch is to introduce a simple general API to count the
memory usage that is independent of any specific caching system. I'm doing this
to "fix" the chicken and egg problem that caches need to know the memory usage,
but we don't really need to count the memory usage without using it for caches.
Implementing caching and memory counting at the same time make both harder than
implementing them one after another.
The main difficulty with counting memory usage is that some memory may be shared
using implicit sharing. We want to avoid double counting such memory. How
exactly shared memory is treated depends a bit on the use case, so no specific
assumptions are made about that in the API. The gathered memory usage is not
expected to be exact. It's expected to be a decent approximation. It's neither a
lower nor an upper bound unless specified by some specific type. Cache systems
generally build on top of heuristics to decide when to free what anyway.
There are two sides to this API:
1. Get the amount of memory used by one or more values. This side is used by
caching systems and/or systems that want to present the used memory to the
user.
2. Tell the caller how much memory is used. This side is used by all kinds of
types that can report their memory usage such as meshes.
```cpp
/* Get how much memory is used by two meshes together. */
MemoryCounter memory;
mesh_a->count_memory(memory);
mesh_b->count_memory(memory);
int64_t bytes_used = memory.counted_bytes();
/* Tell the caller how much memory is used. */
void Mesh::count_memory(blender::MemoryCounter &memory) const
{
memory.add_shared(this->runtime->face_offsets_sharing_info,
this->face_offsets().size_in_bytes());
/* Forward memory counting to lower level types. This should be fairly common. */
CustomData_count_memory(this->vert_data, this->verts_num, memory);
}
void CustomData_count_memory(const CustomData &data,
const int totelem,
blender::MemoryCounter &memory)
{
for (const CustomDataLayer &layer : Span{data.layers, data.totlayer}) {
memory.add_shared(layer.sharing_info, [&](blender::MemoryCounter &shared_memory) {
/* Not quite correct for all types, but this is only a rough approximation anyway. */
const int64_t elem_size = CustomData_get_elem_size(&layer);
shared_memory.add(totelem * elem_size);
});
}
}
```
Pull Request: https://projects.blender.org/blender/blender/pulls/126295
Previously, values for `ID.flag` and `ID.tag` used the prefixes `LIB_` and
`LIB_TAG` respectively. This was somewhat confusing because it's not really
related to libraries in general. This patch changes the prefix to `ID_FLAG_` and
`ID_TAG_`. This makes it more obvious what they correspond to, simplifying code.
Pull Request: https://projects.blender.org/blender/blender/pulls/125811
The change to use generic "capture field on geometry" utilities for this
node and other nodes like it means `AttributeWriter` with its update
tagging isn't being used anymore, the attribute is just being created
with the new values (for some cases anyway). To fix this, call the
attribute provider's update function when creating the attribute too.
This was noted as useful in 130701763b too.
The initialization of curve and point cloud runtime structs is moved
because they now have to be allocated before any attributes are added.
Seems to work OK in basic cases, but needs more work when copying
outside of Main at least.
Note: There is no behavioral changes expected from this commit.
Note that there are at least two known usecases for this change:
* Liboverrides, as with recursive resync and proxies conversion it
often ends up creating 'virtual' linked data that does not actually
exists in the library blend files.
* Complex versionning code (`do_versions_after_setup`) when it needs
to create new IDs (currently handling linked data that way is just not
supported!).
Implements #107847.
This data was 'hidden' away in a util in
`lib_query.cc`, which made it hard to discover and keep up-to-date.
However, as shown by e.g. #108407, critical low-level features in ID
management code, such as remapping, now rely on this information being
valid.
Also simplify `BKE_library_id_can_use_filter_id` and
`BKE_library_id_can_use_idtype` to make them more generic, relying on
IDTypeInfo to retrieve IDtype-specific info.
No behavioral changes expected here.
With this patch, materials are kept intact in simulation zones and bake nodes
without any additional user action.
This implements the design proposed in #108410 to support referencing
data-blocks (only materials for now) in the baked data. The task also describes
why this is not a trivial issue. A previous attempt was implemented in #109703
but it didn't work well-enough.
The solution is to have an explicit `name (+ library name) -> data-block`
mapping that is stored in the modifier for each bake node and simulation zone.
The `library name` is necessary for it to be unique within a .blend file. Note
that this refers to the name of the `Library` data-block and not a file path.
The baked data only contains the names of the used data-blocks. When the baked
data is loaded, the correct material data-block is looked up from the mapping.
### Automatic Mapping Generation
The most tricky aspect of this approach is to make it feel mostly automatic.
From the user point-of-view, it should just work. Therefore, we don't want the
user to have to create the mapping manually in the majority of cases. Creating
the mapping automatically is difficult because the data-blocks that should
become part of the mapping are only known during depsgraph evaluation. So we
somehow have to gather the missing data blocks during evaluation and then write
the new mappings back to the original data.
While writing back to original data is something we do in some cases already,
the situation here is different, because we are actually creating new relations
between data-blocks. This also means that we'll have to do user-counting. Since
user counts in data-blocks are *not* atomic, we can't do that from multiple
threads at the same time. Also, under some circumstances, it may be necessary to
trigger depsgraph evaluation again after the write-back because it actually
affects the result.
To solve this, a small new API is added in `DEG_depsgraph_writeback_sync.hh`. It
allows gathering tasks which write back to original data in a synchronous way
which may also require a reevaluation.
### Accessing the Mapping
A new `BakeDataBlockMap` is passed to geometry nodes evaluation by the modifier.
This map allows getting the `ID` pointer that should be used for a specific
data-block name that is stored in baked data. It's also used to gather all the
missing data mappings during evaluation.
### Weak ID References
The baked/cached geometries may have references to other data-blocks (currently
only materials, but in the future also e.g. instanced objects/collections).
However, the pointers of these data-blocks are not stable over time. That is
especially true when storing/loading the data from disk, but also just when
playing back the animation. Therefore, the used data-blocks have to referenced
in a different way at run-time.
This is solved by adding `std::unique_ptr<bake::BakeMaterialsList>` to the
run-time data of various geometry data-blocks. If the data-block is cached over
a longer period of time (such that material pointers can't be used directly), it
stores the material name (+ library name) used by each material slot. When the
geometry is used again, the material pointers are restored using these weak name
references and the `BakeDataBlockMap`.
### Manual Mapping Management
There is a new `Data-Blocks` panel in the bake settings in the node editor
sidebar that allows inspecting and modifying the data-blocks that are used when
baking. The user can change what data-block a specific name is mapped to.
Pull Request: https://projects.blender.org/blender/blender/pulls/117043
Some common headers were including this. Separating the includes
will ideally lead to better conceptual separation between CustomData
and the attribute API too. Mostly the change is adding the file to
places where it was included indirectly before. But some code is
shuffled around to hopefully better places as well.
Each value is now out of the global namespace, so they can be shorter
and easier to read. Most of this commit just adds the necessary casting
and namespace specification. `enum class` can be forward declared since
it has a specified size. We will make use of that in the next commit.
Implement the next phases of bounds improvement design #96968.
Mainly the following changes:
Don't use `Object.runtime.bb` for performance caching volume bounds.
This is redundant with the cache in most geometry data-block types.
Instead, this becomes `Object.runtime.bounds_eval`, and is only used
where it's actually needed: syncing the bounds from the evaluated
geometry in the active depsgraph to the original object.
Remove all redundant functions to access geometry bounds with an
Object argument. These make the whole design confusing, since they
access geometry bounds at an object level.
Use `std::optional<Bounds<float3>>` to pass and store bounds instead
of an allocated `BoundBox` struct. This uses less space, avoids
small heap allocations, and generally simplifies code, since we
usually only want the min and max anyway.
After this, to avoid performance regressions, we should also cache
bounds in volumes, and maybe the legacy curve and GP data types
(though it might not be worth the effort for those legacy types).
Pull Request: https://projects.blender.org/blender/blender/pulls/114933
Move object runtime data to a separate header and allocate it separately
as `blender::bke::ObjectRuntime`. This is how node, mesh, curves, and
point cloud runtime data is stored.
Benefits:
- Allow using C++ types in object runtime data
- Reduce space required for Object struct in files
- Increase conceptual separation between DNA and runtime data
- Remove the need to add manual padding in runtime data
- Include runtime struct definition only in files that require it
Pull Request: https://projects.blender.org/blender/blender/pulls/113957
