The memfile undo data-block change detection didn't work for meshes
because we ended up writing a new pointer every time. In practice the
array the pointer references is always empty anyway, so we can just add
a check and write null instead.
Unfortunately this fix only applies to 4.5, since the attribute DNA
data (which is actually used at runtime in 5.0) is created temporarily
specifically for writing, so it gets a new address every time.
We'll probably need to solve #127706 in 5.0 to fix this.
Pull Request: https://projects.blender.org/blender/blender/pulls/141457
**Problem Description**
Blender's current mesh data layout often lacks spatial coherence,
causing performance bottlenecks during BVH construction for sculpting
and painting operations. Each time a BVH is built, the system must
recompute spatial partitioning and vertex groupings from scratch,
leading to redundant calculations and suboptimal memory access patterns.
**Proposed Solution**
This patch implements pre-computed spatial organization of mesh data
through a new `mesh_apply_spatial_organization()` function that:
- Reorders vertices and faces based on spatial locality using recursive
spatial partitioning.
- Stores pre-computed MeshGroup hierarchies in MeshRuntime for reuse.
- Enables the BVH system to bypass expensive spatial computation when
pre-organized data is available.
This approach separates the expensive spatial computation from more
frequent BVH rebuilds, providing sustained performance improvements
across multiple sculpting operations.
**Limitations**
- Requires manual invocation (occurs automatically only during remesh
operations).
- Additional memory overhead for storing MeshGroup metadata.
- One-time computational cost during initial organization.
- Spatial group references are not yet stored in files.
**User Interface**
The feature is accessible via a new "Reorder Mesh Spatially" operator in
the Mesh Data Properties panel under the Geometry Data section. Users
can invoke it manually when needed, or it will be applied automatically
during quadriflow and voxel remesh operations. The operator provides
feedback confirming successful spatial reordering.
Pull Request: https://projects.blender.org/blender/blender/pulls/139536
Change `eCustomDataType` to `bke::AttrType` for uses of the attribute
API (the `AttributeAccessor` one anyway). I didn't touch any values that
might be saved in files; those should be handled on a case by case basis.
Part of #122398
Pull Request: https://projects.blender.org/blender/blender/pulls/141301
Remove the use_attribute_storage_write experimental option and always
write in the new format, which is supported by 4.5. The new format is
only used at runtime by point clouds currently but there is no reason
for it to be an option at this point.
This is a second commit repeating 84212bae4b
after that was reverted. Issues that came up with Grease Pencil writing
have been resolved in the meantime.
Remove the `use_attribute_storage_write` experimental option and always
write in the new format, which is supported by 4.5. The new format is
only used at runtime by point clouds currently but there is no reason
for it to be an option at this point.
Pull Request: https://projects.blender.org/blender/blender/pulls/140284
Required for the transition to `AttributeStorage`, because this
information was previously stored in `CustomDataLayer` flags.
This is already the way we store the active/default color attribute status.
Migrating runtime usage to use the names isn't done here, just like
we still use `CustomData` everywhere at runtime for `Mesh`.
Pull Request: https://projects.blender.org/blender/blender/pulls/140137
The transition to `AttributeStorage` requires the active and default
UV map status to be stored separately, since currently they're
stored as flags on `CustomDataLayer`. This commit adds forward
compatible reading for when the status is written as attribute names
on `Mesh` instead, which is what we already do for color attributes.
Pull Request: https://projects.blender.org/blender/blender/pulls/140134
This commit implements #125759.
It removes:
* Blender does not build on big endian systems anymore.
* Support for opening blendfiles written from a big endian system is
removed.
It keeps:
* Support to generate thumbnails from big endian blendfiles.
* BE support in `extern` or `intern` libraries, including Cycles.
* Support to open big endian versions of third party file formats:
- PLY files.
- Some image files (cineon, ...).
Pull Request: https://projects.blender.org/blender/blender/pulls/140138
From what I know, we don't really have a good way to detect whether a material
index is non-sensical in the right places. That's because the same material
index on a mesh may not make sense on one object but can still make sense on
another. This is the issue we fixed in the first place when the regression was
introduced.
What we can do though is to check which exact material indices a mesh is
actually using (not just the maximum). This allows us to skip a lot of work for
unused material indices. This doesn't help when a mesh has thousands of unique
non-sensical material indices, but it should be an improvement in the majority
of cases.
This patch adds a cache of used material indices to `Mesh`. The drawing code
requests that cache if the maximum material index is above some threshold (16
currently). We don't want to compute it all the time, because it requires
iterating over the mesh (at least once, then it is cached). So it's only worth
the extra cost of the there is at least one large material index. The threshold
also ensure that the large majority of scenes is not affected by this patch
performance wise.
Pull Request: https://projects.blender.org/blender/blender/pulls/139781
Complete the versioning and writing changes from these commits:
- f9b627d29c
- f2bcd73bd2
- c5ba8bd7c2
4.5 has the forward compatibility handling to support reading files
written by 5.0. This removes the conversion when writing files
and removes the lookup of ".uv_seam" as a fallback for "uv_seam".
Pull Request: https://projects.blender.org/blender/blender/pulls/139841
Check if either the memory is zero or already matches the default value,
and copy. This simplifies a common pattern to a single line.
Preparing for default initializers in DNA (#134531).
Pull Request: https://projects.blender.org/blender/blender/pulls/138830
As described in #122398, implement read and write support for a new
attribute storage system. Currently this is only implemented to support
forward compatibility; the format used at runtime isn't changed at all.
That can be done one step at a time during the 4.5 and 5.0 development
cycles. A new experimental option for testing tells Blender to always
save with the new format.
The main benefit of the new structure is that it matches the attribute
system design, it allows for future attribute storage optimization, and
each attribute is an allocated struct, which will give pointer stability
for the Python API.
The next step is to connect the attribute API and the RNA API to
AttributeStorage for the simplest geometry type, point clouds.
Pull Request: https://projects.blender.org/blender/blender/pulls/133874
This adds a version of `BKE_id_new_nomain` that takes the ID type parameter as
template argument. This allows the function the return the newly created ID with
the correct type, removing the need to use `static_cast` on the call-site.
To make this work, I added a static `id_type` member to every ID struct. This
can also be used to create a similar API for other id management functions in
future patches.
```cpp
// Old
Mesh *mesh = static_cast<Mesh *>(BKE_id_new_nomain(ID_ME, "Mesh"));
// New
Mesh *mesh = BKE_id_new_nomain<Mesh>("Mesh");
```
Pull Request: https://projects.blender.org/blender/blender/pulls/138383
The goal here is to avoid having to cast to and from `ID` when getting the
evaluated or original ID using the depsgraph API, which is often verbose and not
type safe. To solve this, there are now `DEG_get_original` and
`DEG_get_evaluated` methods which are templated on the type and use a new
`is_ID_v` static type check to make sure it's only used with valid types.
This allows removing quite some verbosity on all the call sites. I also removed
`DEG_get_original_object`, because that does not have to be a special case
anymore.
Pull Request: https://projects.blender.org/blender/blender/pulls/137629
Currently this code uses the `Mesh` struct as a value embedded in the
undo step struct. This causes the need for various hacky workarounds and
abstraction leaks. It's simpler to just allocate it in the regular way
The Shape Key ID can also be freed with the regular ID free function,
avoiding the need to expose the internal free function.
Pull Request: https://projects.blender.org/blender/blender/pulls/137613
Clearing a mesh's geometry without freeing the mesh has the requirement
that the mesh remain initialized afterwards. Freeing the mesh doesn't
have that requirement and corresponds to a destructor rather than move
construction with the default value. Avoiding conflating the two
operations simplifies some future feature additions (including #122398).
Add a "dumb vector" storage option for custom normals, with the
"custom_normal" attribute. Adjust the mesh normals caching to
provide this attribute if it's available, and add a geometry node to
store custom normals.
## Free Normals
They're called "free" in the sense that they're just direction vectors
in the object's local space, rather than the existing "smooth corner
fan space" storage. They're also "free" in that they make further
normals calculation very inexpensive, since we just use the custom
normals instead. That's a big improvement from the existing custom
normals storage, which usually significantly decreases
viewport performance. For example, in a simple test file just storing
the vertex normals on a UV sphere, using free normals gives 25 times
better playback performance and 10% lower memory usage.
Free normals are adjusted when applying a transformation to the entire
mesh or when realizing instances, but in general they're not updated for
vertex deformations.
## Set Mesh Normal Node
The new geometry node allows storing free custom normals as well as
the existing corner fan space normals. When free normals are chosen,
free normals can be stored on vertices, faces, or face corners. Using
the face corner domain is necessary to bake existing mixed sharp and
smooth edges into the custom normal vectors.
The node also has a mode for storing edge and mesh sharpness, meant
as a "soft" replacement to the "Set Shade Smooth" node that's a bit
more convenient.
## Normal Input Node
The normal node outputs free custom normals mixed to whatever domain is
requested. A "true normal" output that ignores custom normals and
sharpness is added as well.
Across Blender, custom normals are generally accessed via face and
vertex normals, when "true normals" are not requested explicitly.
In many cases that means they are mixed from the face corner domain.
## Future Work
1. There are many places where propagation of free normals could be
improved. They should probably be normalized after mixing, and it
may be useful to not just use 0 vectors for new elements. To keep
the scope of this change smaller, that sort of thing generally isn't
handled here. Searching `CD_NORMAL` gives a hint of where better
propagation could be useful.
2. Free normals are displayed properly in edit mode, but the existing
custom normal editing operators don't work with free normals yet.
This will hopefully be fairly straightforward since custom normals
are usually converted to `float3` for editing anyway. Edit mode
changes aren't included here because they're unnecessary for the
procedural custom normals use cases.
3. Most importers can probably switch to using free normals instead,
or at least provide an option for it. That will give a significant
import performance improvement, and an improvement of Blender's
FPS for imported scenes too.
Pull Request: https://projects.blender.org/blender/blender/pulls/132583
Due to !133498 layers containing incorrect material indices could lead to generating
to many GPU resources crashing Blender. As there is a maximum number of material slots
we can limit it.
This is just a safeguard for incorrect values in material_index attribute layer. Working with
this material index attributes should automatically limit, but I leave that to the modeling
module.
Pull Request: https://projects.blender.org/blender/blender/pulls/136302
The main issue of 'type-less' standard C allocations is that there is no check on
allocated type possible.
This is a serious source of annoyance (and crashes) when making some
low-level structs non-trivial, as tracking down all usages of these
structs in higher-level other structs and their allocation is... really
painful.
MEM_[cm]allocN<T> templates on the other hand do check that the
given type is trivial, at build time (static assert), which makes such issue...
trivial to catch.
NOTE: New code should strive to use MEM_new (i.e. allocation and
construction) as much as possible, even for trivial PoD types.
Pull Request: https://projects.blender.org/blender/blender/pulls/136134
* Fix mesh_copy_data setting the subd wrapper type without copying the
associated tessellated mesh. It's unclear if this currently causes any
issues, but with future changes it would crash.
* Use double checked lock to avoid mutex locks on every call to
BKE_mesh_wrapper_ensure_mdata and BKE_mesh_wrapper_ensure_subdivision.
* Set wrapper type to subd also if subdivision was skipped due to bad
topology or level set to zero. Previous it would do mutex locking and
TBB task isolation on every function call.
* Add assert when BKE_mesh_wrapper_ensure_mdata changes the wrapper type
from subd to mdata. This doesn't really make any sense, it just loses
the subdivision detail. If any code relies on this we need to understand
why and maybe change it.
Pull Request: https://projects.blender.org/blender/blender/pulls/135895
This adds a new `Visual Geometry to Objects` operator which is added to the
`Apply` menu.
It creates new objects from the evaluated geometry of the active object. Other
than e.g. applying modifiers, this operator does keep all generated data and
creates new objects and collections for instances. It does have some
similarities to `Make Instances Real` with the main difference that it doesn't
actually realize instances, i.e. instanced geometry is still shared between all
objects using it.
It does intentionally not replace the original object, because the semantics of
that aren't all that obvious. It's especially tricky when the object was
referenced by other objects. Instance attributes are not preserved currently.
Pull Request: https://projects.blender.org/blender/blender/pulls/134119
Previously we generally expected CustomData layers to have implicit
sharing info, but we didn't require it. This PR clarifies that we do
require layers with non-null data to have implicit sharing info. This
generally makes code simpler because we don't have to have a separate
code path for non-shared layers. For example, it makes the "totelem"
arguments for layer freeing functions unnecessary, since shared data
knows how to free itself. Those arguments are removed in this PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/134578
Replace `const char *` with `StringRef` for the API in `BKE_attribute.h`.
The benefits are slightly simpler code and possibly slightly improved
performance through avoiding the need to measure string length.
Pull Request: https://projects.blender.org/blender/blender/pulls/134183
The issue was that the function that computes the maximum used material index
did not take the edit mesh into account yet. This led the drawing code to
believe that only the first material is used by the mesh.
Pull Request: https://projects.blender.org/blender/blender/pulls/133711
The core issue was that the geometry batch cache (e.g. `MeshBatchCache` or
`PointCloudBatchCache`) was dependent on the object. This is problematic when
the the same geometry is used with multiple different objects because the cache
can't be consistent with all of them.
Fortunately, the only thing that was retrieved from the object was the number of
material slots, so if that can be avoided we should be fine. We can't just use
the number of material slots stored on the geometry because that may have no
material slots but still has material indices which are overridden on the object
level.
The solution is to take make the number of materials for a geometry only
dependent on the actual `material_index` attribute and not on the number of
available slots. More specifically, we find the maximal referenced material
index and handle that many materials. This number does not depend on how many
material slots there are on the object, but it still allows the object to
override materials slots that the mesh references.
A downside is that the maximum material index has to be computed which often
requires an iteration over the mesh. Fortunately, we can cache that quite easily
and the computation can be done in parallel. Also we are probably able to
eagerly update the material index in many cases when it's set instead of
computing it lazily. That is not implemented in this patch though.
The largest part of the patch is making the maximal material index easily
available on all the geometry types. Besides that, the material API is slightly
replaced and the drawing code now makes use of the updated API.
Pull Request: https://projects.blender.org/blender/blender/pulls/133498
Regression in [0] that clamped the shape-key index when exiting
edit-mode but not when entering, causing the "Basis" shape not to
be treated as active when entering edit-mode, then treated as active
when exiting - overwriting it's values from the mesh.
Resolve by clamping on edit-mode enter & exit.
[0]: 225e3460d0
When moving UV seams from special custom data types to attributes,
I considered them similar to mesh selection or visibility which are
"internal" attributes that aren't accessible procedurally and are
hidden from the UI. In retrospect that was the wrong decision; users
expect UV seams to behave more like bevel weights, and that makes sense.
This PR makes UV seams accessible in modifiers (geometry nodes) by
removing the leading period from the attribute name that indicated their
internal status.
The change of the attribute name is a breaking change of the API to some
extent, even though it's technically only mesh data. To mitigate that
issue, the `mesh.attributes["name"]` lookup function is modified to
support both the old and new names. Versioning code renames the
attribute to the new name when loading older files, and renames the
new name to the old name when saving files. That handling will be
removed as a breaking change in 5.0.
Pull Request: https://projects.blender.org/blender/blender/pulls/129803
Move `CD_CUSTOMLOOPNORMAL` to the newly added
`CD_PROP_INT16_2D` generic attribute type. This is similar to
previous commits moving specific custom data types.
The attribute name is `custom_normal`. When the attribute with
that name is on the face corner domain, the code will interpret it
as stored in the existing deformation-invariant spherical coordinate
space.
The API remains the same, with the additional opportunity to edit
custom normal data as an attribute directly (which admittedly is fairly
unintuitive currently).
See #130484.
Pull Request: https://projects.blender.org/blender/blender/pulls/130689
Avoid rebuilding BVH trees when meshes are copied.
Similar to the other uses of the shared cache system,
this can arbitrarily improve performance when meshes
are copied but not deformed and BVH building is the
main bottleneck. In a simple test file I got a 6x speedup.
The amount of code is also reduced and the system is
much simpler overall-- built out of common threading
patterns like `SharedCache` with its double-checked lock.
RAII is used in a few places to simplify memory management
too.
The downside is storing more `SharedCache` items in the
mesh runtime struct. That has a slight cost when copying
a small mesh many times, but we have ideas to improve that
in the future anyway (#104327).
Pull Request: https://projects.blender.org/blender/blender/pulls/130865
Currently replacing the Mesh during evaluation with the object info node
can cause us to use invalid original indices when the source object is
in edit mode. This is really a more fundamental problem though: we have
no way to tell whether an evaluated mesh actually corresponds to the
object's original mesh.
This commit changes to explicitly propagating the edit mesh pointer
through copied and changed meshes during modifier and nodes evaluation,
instead of just blindly copying the edit mode pointer from the original
mesh to the evaluated mesh. A benefit of not writing to the evaluated
mesh means it can be shared, potentially offering a future performance
improvement for uses of the object info node.
When we detect an invalid correspondence between the evaluated/original
meshes, we skip extracting the cage mesh's data and skip extracting edit
mesh data from the evaluated object.
This commit also moves the source of "object is in edit mode" truth in
the draw module from whether the evaluated mesh has an edit mode pointer
to the object's mode flag. That's a simplification that's also helpful
to reduce the strong linking between BMesh and edit mode.
Pull Request: https://projects.blender.org/blender/blender/pulls/120999
Previously, the attribute accessor were defined in the `geometry_component_*.cc`
files. This made sense back in the day, because this attribute API was only used
through `GeometryComponent`. However, nowadays this attribute API is independent
of `GeometryComponent`. E.g. one can use `mesh.attributes()` without ever
creating a component.
The refactor contains the following changes:
* Move attribute accessors to separate files for each geometry type. E.g. from
`geometry_component_mesh.cc` to `mesh_attributes.cc`.
* Move implementations of e.g. `Mesh::attributes()` to `mesh.cc`.
* Provide access to the `AttributeAccessorFunctions` without actually having a
geometry. This will be useful to e.g. implement
`attribute_is_builtin_on_component_type` without dummy components.
Pull Request: https://projects.blender.org/blender/blender/pulls/130516
Previously, the `AttributeIDRef` wrapper was needed because it also had to
contain a pointer to an `AnonymousAttributeID`. However, since
b279a6d703 this is not necessary anymore.
Therefore we can use "raw" `StringRef` now which reduces the mental overhead
when working with attributes and also simplifies code.
Pull Request: https://projects.blender.org/blender/blender/pulls/127140
And same for the `copy_layout` function. These functions do not free any
potentially existing data in destination, but expect it to be uninitialized.
Hopefully these new names will make it more clear and avoid bugs like #122160.
We often have the situation where it would be good if we could easily estimate
the memory usage of some value (e.g. a mesh, or volume). Examples of where we
ran into this in the past:
* Undo step size.
* Caching of volume grids.
* Caching of loaded geometries for import geometry nodes.
Generally, most caching systems would benefit from the ability to know how much
memory they currently use to make better decisions about which data to free and
when. The goal of this patch is to introduce a simple general API to count the
memory usage that is independent of any specific caching system. I'm doing this
to "fix" the chicken and egg problem that caches need to know the memory usage,
but we don't really need to count the memory usage without using it for caches.
Implementing caching and memory counting at the same time make both harder than
implementing them one after another.
The main difficulty with counting memory usage is that some memory may be shared
using implicit sharing. We want to avoid double counting such memory. How
exactly shared memory is treated depends a bit on the use case, so no specific
assumptions are made about that in the API. The gathered memory usage is not
expected to be exact. It's expected to be a decent approximation. It's neither a
lower nor an upper bound unless specified by some specific type. Cache systems
generally build on top of heuristics to decide when to free what anyway.
There are two sides to this API:
1. Get the amount of memory used by one or more values. This side is used by
caching systems and/or systems that want to present the used memory to the
user.
2. Tell the caller how much memory is used. This side is used by all kinds of
types that can report their memory usage such as meshes.
```cpp
/* Get how much memory is used by two meshes together. */
MemoryCounter memory;
mesh_a->count_memory(memory);
mesh_b->count_memory(memory);
int64_t bytes_used = memory.counted_bytes();
/* Tell the caller how much memory is used. */
void Mesh::count_memory(blender::MemoryCounter &memory) const
{
memory.add_shared(this->runtime->face_offsets_sharing_info,
this->face_offsets().size_in_bytes());
/* Forward memory counting to lower level types. This should be fairly common. */
CustomData_count_memory(this->vert_data, this->verts_num, memory);
}
void CustomData_count_memory(const CustomData &data,
const int totelem,
blender::MemoryCounter &memory)
{
for (const CustomDataLayer &layer : Span{data.layers, data.totlayer}) {
memory.add_shared(layer.sharing_info, [&](blender::MemoryCounter &shared_memory) {
/* Not quite correct for all types, but this is only a rough approximation anyway. */
const int64_t elem_size = CustomData_get_elem_size(&layer);
shared_memory.add(totelem * elem_size);
});
}
}
```
Pull Request: https://projects.blender.org/blender/blender/pulls/126295
Previously, values for `ID.flag` and `ID.tag` used the prefixes `LIB_` and
`LIB_TAG` respectively. This was somewhat confusing because it's not really
related to libraries in general. This patch changes the prefix to `ID_FLAG_` and
`ID_TAG_`. This makes it more obvious what they correspond to, simplifying code.
Pull Request: https://projects.blender.org/blender/blender/pulls/125811
All of these changes should be trivial, like using `string` read/write
code for strings, and a convert a few usages of 'raw data' read/write
calls to the 'struct' ones.
No behavioral changes expected here.
