Files
test2/source/blender/blenkernel/intern/geometry_component_mesh.cc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

192 lines
5.0 KiB
C++
Raw Normal View History

/* SPDX-FileCopyrightText: 2023 Blender Authors
*
* SPDX-License-Identifier: GPL-2.0-or-later */
#include "BLI_math_vector.hh"
#include "BKE_geometry_fields.hh"
#include "BKE_geometry_set.hh"
2024-01-15 12:44:04 -05:00
#include "BKE_lib_id.hh"
#include "BKE_mesh.hh"
#include "attribute_access_intern.hh"
namespace blender::bke {
/* -------------------------------------------------------------------- */
/** \name Geometry Component Implementation
* \{ */
MeshComponent::MeshComponent() : GeometryComponent(Type::Mesh) {}
MeshComponent::MeshComponent(Mesh *mesh, GeometryOwnershipType ownership)
: GeometryComponent(Type::Mesh), mesh_(mesh), ownership_(ownership)
{
}
MeshComponent::~MeshComponent()
{
this->clear();
}
GeometryComponentPtr MeshComponent::copy() const
{
MeshComponent *new_component = new MeshComponent();
if (mesh_ != nullptr) {
new_component->mesh_ = BKE_mesh_copy_for_eval(*mesh_);
new_component->ownership_ = GeometryOwnershipType::Owned;
}
return GeometryComponentPtr(new_component);
}
void MeshComponent::clear()
{
BLI: support weak users and version in implicit sharing info The main goal of these changes is to support checking if some data has been changed over time. This is used by the WIP simulation nodes during baking to detect which attributes have to be stored in every frame because they have changed. By using a combination of a weak user count and a version counter, it is possible to detect that an attribute (or any data controlled by implicit sharing) has not been changed with O(1) memory and time. It's still possible that the data has been changed multiple times and is the same in the end and beginning of course. That wouldn't be detected using this mechanism. The `ImplicitSharingInfo` struct has a new weak user count. A weak reference is one that does not keep the referenced data alive, but makes sure that the `ImplicitSharingInfo` itself is not deleted. If some piece of data has one strong and multiple weak users, it is still mutable. If the strong user count goes down to zero, the referenced data is freed. Remaining weak users can check for this condition using `is_expired`. This is a bit similar to `std::weak_ptr` but there is an important difference: a weak user can not become a strong user while one can create a `shared_ptr` from a `weak_ptr`. This restriction is necessary, because some code might be changing the referenced data assuming that it is the only owner. If another thread suddenly adds a new owner, the data would be shared again and the first thread would not have been allowed to modify the data in the first place. There is also a new integer version counter in `ImplicitSharingInfo`. It is incremented whenever some code wants to modify the referenced data. Obviously, this can only be done when the data is not shared because then it would be immutable. By comparing an old and new version number of the same sharing info, one can check if the data has been modified. One has to keep a weak reference to the sharing info together with the old version number to ensure that the new sharing info is still the same as the old one. Without this, it can happen that the sharing info was freed and a new one was allocated at the same pointer address. Using a strong reference for this purpose does not work, because then the data would never be modified because it's shared.
2023-04-28 12:03:42 +02:00
BLI_assert(this->is_mutable() || this->is_expired());
if (mesh_ != nullptr) {
if (ownership_ == GeometryOwnershipType::Owned) {
BKE_id_free(nullptr, mesh_);
}
mesh_ = nullptr;
}
}
bool MeshComponent::has_mesh() const
{
return mesh_ != nullptr;
}
void MeshComponent::replace(Mesh *mesh, GeometryOwnershipType ownership)
{
BLI_assert(this->is_mutable());
this->clear();
mesh_ = mesh;
ownership_ = ownership;
}
Mesh *MeshComponent::release()
{
BLI_assert(this->is_mutable());
Mesh *mesh = mesh_;
mesh_ = nullptr;
return mesh;
}
const Mesh *MeshComponent::get() const
{
return mesh_;
}
Mesh *MeshComponent::get_for_write()
{
BLI_assert(this->is_mutable());
if (ownership_ == GeometryOwnershipType::ReadOnly) {
mesh_ = BKE_mesh_copy_for_eval(*mesh_);
ownership_ = GeometryOwnershipType::Owned;
}
return mesh_;
}
bool MeshComponent::is_empty() const
{
return mesh_ == nullptr;
}
bool MeshComponent::owns_direct_data() const
{
return ownership_ == GeometryOwnershipType::Owned;
}
void MeshComponent::ensure_owns_direct_data()
{
BLI_assert(this->is_mutable());
if (ownership_ != GeometryOwnershipType::Owned) {
if (mesh_) {
mesh_ = BKE_mesh_copy_for_eval(*mesh_);
}
ownership_ = GeometryOwnershipType::Owned;
}
}
Core: introduce MemoryCounter API We often have the situation where it would be good if we could easily estimate the memory usage of some value (e.g. a mesh, or volume). Examples of where we ran into this in the past: * Undo step size. * Caching of volume grids. * Caching of loaded geometries for import geometry nodes. Generally, most caching systems would benefit from the ability to know how much memory they currently use to make better decisions about which data to free and when. The goal of this patch is to introduce a simple general API to count the memory usage that is independent of any specific caching system. I'm doing this to "fix" the chicken and egg problem that caches need to know the memory usage, but we don't really need to count the memory usage without using it for caches. Implementing caching and memory counting at the same time make both harder than implementing them one after another. The main difficulty with counting memory usage is that some memory may be shared using implicit sharing. We want to avoid double counting such memory. How exactly shared memory is treated depends a bit on the use case, so no specific assumptions are made about that in the API. The gathered memory usage is not expected to be exact. It's expected to be a decent approximation. It's neither a lower nor an upper bound unless specified by some specific type. Cache systems generally build on top of heuristics to decide when to free what anyway. There are two sides to this API: 1. Get the amount of memory used by one or more values. This side is used by caching systems and/or systems that want to present the used memory to the user. 2. Tell the caller how much memory is used. This side is used by all kinds of types that can report their memory usage such as meshes. ```cpp /* Get how much memory is used by two meshes together. */ MemoryCounter memory; mesh_a->count_memory(memory); mesh_b->count_memory(memory); int64_t bytes_used = memory.counted_bytes(); /* Tell the caller how much memory is used. */ void Mesh::count_memory(blender::MemoryCounter &memory) const { memory.add_shared(this->runtime->face_offsets_sharing_info, this->face_offsets().size_in_bytes()); /* Forward memory counting to lower level types. This should be fairly common. */ CustomData_count_memory(this->vert_data, this->verts_num, memory); } void CustomData_count_memory(const CustomData &data, const int totelem, blender::MemoryCounter &memory) { for (const CustomDataLayer &layer : Span{data.layers, data.totlayer}) { memory.add_shared(layer.sharing_info, [&](blender::MemoryCounter &shared_memory) { /* Not quite correct for all types, but this is only a rough approximation anyway. */ const int64_t elem_size = CustomData_get_elem_size(&layer); shared_memory.add(totelem * elem_size); }); } } ``` Pull Request: https://projects.blender.org/blender/blender/pulls/126295
2024-08-15 10:54:21 +02:00
void MeshComponent::count_memory(MemoryCounter &memory) const
{
if (mesh_) {
mesh_->count_memory(memory);
}
}
/** \} */
/* -------------------------------------------------------------------- */
/** \name Mesh Normals Field Input
* \{ */
VArray<float3> mesh_normals_varray(const Mesh &mesh,
BLI: refactor IndexMask for better performance and memory usage Goals of this refactor: * Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an `int64_t` for each index which is more than necessary in pretty much all practical cases currently. Using `int32_t` might still become limiting in the future in case we use this to index e.g. byte buffers larger than a few gigabytes. We also don't want to template `IndexMask`, because that would cause a split in the "ecosystem", or everything would have to be implemented twice or templated. * Allow for more multi-threading. The old `IndexMask` contains a single array. This is generally good but has the problem that it is hard to fill from multiple-threads when the final size is not known from the beginning. This is commonly the case when e.g. converting an array of bool to an index mask. Currently, this kind of code only runs on a single thread. * Allow for efficient set operations like join, intersect and difference. It should be possible to multi-thread those operations. * It should be possible to iterate over an `IndexMask` very efficiently. The most important part of that is to avoid all memory access when iterating over continuous ranges. For some core nodes (e.g. math nodes), we generate optimized code for the cases of irregular index masks and simple index ranges. To achieve these goals, a few compromises had to made: * Slicing of the mask (at specific indices) and random element access is `O(log #indices)` now, but with a low constant factor. It should be possible to split a mask into n approximately equally sized parts in `O(n)` though, making the time per split `O(1)`. * Using range-based for loops does not work well when iterating over a nested data structure like the new `IndexMask`. Therefor, `foreach_*` functions with callbacks have to be used. To avoid extra code complexity at the call site, the `foreach_*` methods support multi-threading out of the box. The new data structure splits an `IndexMask` into an arbitrary number of ordered `IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The indices within a segment are stored as `int16_t`. Each segment has an additional `int64_t` offset which allows storing arbitrary `int64_t` indices. This approach has the main benefits that segments can be processed/constructed individually on multiple threads without a serial bottleneck. Also it reduces the memory requirements significantly. For more details see comments in `BLI_index_mask.hh`. I did a few tests to verify that the data structure generally improves performance and does not cause regressions: * Our field evaluation benchmarks take about as much as before. This is to be expected because we already made sure that e.g. add node evaluation is vectorized. The important thing here is to check that changes to the way we iterate over the indices still allows for auto-vectorization. * Memory usage by a mask is about 1/4 of what it was before in the average case. That's mainly caused by the switch from `int64_t` to `int16_t` for indices. In the worst case, the memory requirements can be larger when there are many indices that are very far away. However, when they are far away from each other, that indicates that there aren't many indices in total. In common cases, memory usage can be way lower than 1/4 of before, because sub-ranges use static memory. * For some more specific numbers I benchmarked `IndexMask::from_bools` in `index_mask_from_selection` on 10.000.000 elements at various probabilities for `true` at every index: ``` Probability Old New 0 4.6 ms 0.8 ms 0.001 5.1 ms 1.3 ms 0.2 8.4 ms 1.8 ms 0.5 15.3 ms 3.0 ms 0.8 20.1 ms 3.0 ms 0.999 25.1 ms 1.7 ms 1 13.5 ms 1.1 ms ``` Pull Request: https://projects.blender.org/blender/blender/pulls/104629
2023-05-24 18:11:41 +02:00
const IndexMask &mask,
const AttrDomain domain,
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
const bool no_corner_normals,
const bool true_normals)
{
switch (domain) {
case AttrDomain::Face: {
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
return VArray<float3>::ForSpan(true_normals ? mesh.face_normals_true() :
mesh.face_normals());
}
case AttrDomain::Point: {
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
return VArray<float3>::ForSpan(true_normals ? mesh.vert_normals_true() :
mesh.vert_normals());
}
case AttrDomain::Edge: {
/* In this case, start with vertex normals and convert to the edge domain, since the
Refactor: Move normals out of MVert, lazy calculation As described in T91186, this commit moves mesh vertex normals into a contiguous array of float vectors in a custom data layer, how face normals are currently stored. The main interface is documented in `BKE_mesh.h`. Vertex and face normals are now calculated on-demand and cached, retrieved with an "ensure" function. Since the logical state of a mesh is now "has normals when necessary", they can be retrieved from a `const` mesh. The goal is to use on-demand calculation for all derived data, but leave room for eager calculation for performance purposes (modifier evaluation is threaded, but viewport data generation is not). **Benefits** This moves us closer to a SoA approach rather than the current AoS paradigm. Accessing a contiguous `float3` is much more efficient than retrieving data from a larger struct. The memory requirements for accessing only normals or vertex locations are smaller, and at the cost of more memory usage for just normals, they now don't have to be converted between float and short, which also simplifies code In the future, the remaining items can be removed from `MVert`, leaving only `float3`, which has similar benefits (see T93602). Removing the combination of derived and original data makes it conceptually simpler to only calculate normals when necessary. This is especially important now that we have more opportunities for temporary meshes in geometry nodes. **Performance** In addition to the theoretical future performance improvements by making `MVert == float3`, I've done some basic performance testing on this patch directly. The data is fairly rough, but it gives an idea about where things stand generally. - Mesh line primitive 4m Verts: 1.16x faster (36 -> 31 ms), showing that accessing just `MVert` is now more efficient. - Spring Splash Screen: 1.03-1.06 -> 1.06-1.11 FPS, a very slight change that at least shows there is no regression. - Sprite Fright Snail Smoosh: 3.30-3.40 -> 3.42-3.50 FPS, a small but observable speedup. - Set Position Node with Scaled Normal: 1.36x faster (53 -> 39 ms), shows that using normals in geometry nodes is faster. - Normal Calculation 1.6m Vert Cube: 1.19x faster (25 -> 21 ms), shows that calculating normals is slightly faster now. - File Size of 1.6m Vert Cube: 1.03x smaller (214.7 -> 208.4 MB), Normals are not saved in files, which can help with large meshes. As for memory usage, it may be slightly more in some cases, but I didn't observe any difference in the production files I tested. **Tests** Some modifiers and cycles test results need to be updated with this commit, for two reasons: - The subdivision surface modifier is not responsible for calculating normals anymore. In master, the modifier creates different normals than the result of the `Mesh` normal calculation, so this is a bug fix. - There are small differences in the results of some modifiers that use normals because they are not converted to and from `short` anymore. **Future improvements** - Remove `ModifierTypeInfo::dependsOnNormals`. Code in each modifier already retrieves normals if they are needed anyway. - Copy normals as part of a better CoW system for attributes. - Make more areas use lazy instead of eager normal calculation. - Remove `BKE_mesh_normals_tag_dirty` in more places since that is now the default state of a new mesh. - Possibly apply a similar change to derived face corner normals. Differential Revision: https://developer.blender.org/D12770
2022-01-13 14:37:58 -06:00
* conversion from edges to vertices is very simple. Use "manual" domain interpolation
* instead of the GeometryComponent API to avoid calculating unnecessary values and to
* allow normalizing the result more simply. */
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
Span<float3> vert_normals = true_normals ? mesh.vert_normals_true() : mesh.vert_normals();
Mesh: Move edges to a generic attribute Implements #95966, as the final step of #95965. This commit changes the storage of mesh edge vertex indices from the `MEdge` type to the generic `int2` attribute type. This follows the general design for geometry and the attribute system, where the data storage type and the usage semantics are separated. The main benefit of the change is reduced memory usage-- the requirements of storing mesh edges is reduced by 1/3. For example, this saves 8MB on a 1 million vertex grid. This also gives performance benefits to any memory-bound mesh processing algorithm that uses edges. Another benefit is that all of the edge's vertex indices are contiguous. In a few cases, it's helpful to process all of them as `Span<int>` rather than `Span<int2>`. Similarly, the type is more likely to match a generic format used by a library, or code that shouldn't know about specific Blender `Mesh` types. Various Notes: - The `.edge_verts` name is used to reflect a mapping between domains, similar to `.corner_verts`, etc. The period means that it the data shouldn't change arbitrarily by the user or procedural operations. - `edge[0]` is now used instead of `edge.v1` - Signed integers are used instead of unsigned to reduce the mixing of signed-ness, which can be error prone. - All of the previously used core mesh data types (`MVert`, `MEdge`, `MLoop`, `MPoly` are now deprecated. Only generic types are used). - The `vec2i` DNA type is used in the few C files where necessary. Pull Request: https://projects.blender.org/blender/blender/pulls/106638
2023-04-17 13:47:41 +02:00
const Span<int2> edges = mesh.edges();
Array<float3> edge_normals(mask.min_array_size());
BLI: refactor IndexMask for better performance and memory usage Goals of this refactor: * Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an `int64_t` for each index which is more than necessary in pretty much all practical cases currently. Using `int32_t` might still become limiting in the future in case we use this to index e.g. byte buffers larger than a few gigabytes. We also don't want to template `IndexMask`, because that would cause a split in the "ecosystem", or everything would have to be implemented twice or templated. * Allow for more multi-threading. The old `IndexMask` contains a single array. This is generally good but has the problem that it is hard to fill from multiple-threads when the final size is not known from the beginning. This is commonly the case when e.g. converting an array of bool to an index mask. Currently, this kind of code only runs on a single thread. * Allow for efficient set operations like join, intersect and difference. It should be possible to multi-thread those operations. * It should be possible to iterate over an `IndexMask` very efficiently. The most important part of that is to avoid all memory access when iterating over continuous ranges. For some core nodes (e.g. math nodes), we generate optimized code for the cases of irregular index masks and simple index ranges. To achieve these goals, a few compromises had to made: * Slicing of the mask (at specific indices) and random element access is `O(log #indices)` now, but with a low constant factor. It should be possible to split a mask into n approximately equally sized parts in `O(n)` though, making the time per split `O(1)`. * Using range-based for loops does not work well when iterating over a nested data structure like the new `IndexMask`. Therefor, `foreach_*` functions with callbacks have to be used. To avoid extra code complexity at the call site, the `foreach_*` methods support multi-threading out of the box. The new data structure splits an `IndexMask` into an arbitrary number of ordered `IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The indices within a segment are stored as `int16_t`. Each segment has an additional `int64_t` offset which allows storing arbitrary `int64_t` indices. This approach has the main benefits that segments can be processed/constructed individually on multiple threads without a serial bottleneck. Also it reduces the memory requirements significantly. For more details see comments in `BLI_index_mask.hh`. I did a few tests to verify that the data structure generally improves performance and does not cause regressions: * Our field evaluation benchmarks take about as much as before. This is to be expected because we already made sure that e.g. add node evaluation is vectorized. The important thing here is to check that changes to the way we iterate over the indices still allows for auto-vectorization. * Memory usage by a mask is about 1/4 of what it was before in the average case. That's mainly caused by the switch from `int64_t` to `int16_t` for indices. In the worst case, the memory requirements can be larger when there are many indices that are very far away. However, when they are far away from each other, that indicates that there aren't many indices in total. In common cases, memory usage can be way lower than 1/4 of before, because sub-ranges use static memory. * For some more specific numbers I benchmarked `IndexMask::from_bools` in `index_mask_from_selection` on 10.000.000 elements at various probabilities for `true` at every index: ``` Probability Old New 0 4.6 ms 0.8 ms 0.001 5.1 ms 1.3 ms 0.2 8.4 ms 1.8 ms 0.5 15.3 ms 3.0 ms 0.8 20.1 ms 3.0 ms 0.999 25.1 ms 1.7 ms 1 13.5 ms 1.1 ms ``` Pull Request: https://projects.blender.org/blender/blender/pulls/104629
2023-05-24 18:11:41 +02:00
mask.foreach_index([&](const int i) {
Mesh: Move edges to a generic attribute Implements #95966, as the final step of #95965. This commit changes the storage of mesh edge vertex indices from the `MEdge` type to the generic `int2` attribute type. This follows the general design for geometry and the attribute system, where the data storage type and the usage semantics are separated. The main benefit of the change is reduced memory usage-- the requirements of storing mesh edges is reduced by 1/3. For example, this saves 8MB on a 1 million vertex grid. This also gives performance benefits to any memory-bound mesh processing algorithm that uses edges. Another benefit is that all of the edge's vertex indices are contiguous. In a few cases, it's helpful to process all of them as `Span<int>` rather than `Span<int2>`. Similarly, the type is more likely to match a generic format used by a library, or code that shouldn't know about specific Blender `Mesh` types. Various Notes: - The `.edge_verts` name is used to reflect a mapping between domains, similar to `.corner_verts`, etc. The period means that it the data shouldn't change arbitrarily by the user or procedural operations. - `edge[0]` is now used instead of `edge.v1` - Signed integers are used instead of unsigned to reduce the mixing of signed-ness, which can be error prone. - All of the previously used core mesh data types (`MVert`, `MEdge`, `MLoop`, `MPoly` are now deprecated. Only generic types are used). - The `vec2i` DNA type is used in the few C files where necessary. Pull Request: https://projects.blender.org/blender/blender/pulls/106638
2023-04-17 13:47:41 +02:00
const int2 &edge = edges[i];
BLI: Refactor vector types & functions to use templates This patch implements the vector types (i.e:`float2`) by making heavy usage of templating. All vector functions are now outside of the vector classes (inside the `blender::math` namespace) and are not vector size dependent for the most part. In the ongoing effort to make shaders less GL centric, we are aiming to share more code between GLSL and C++ to avoid code duplication. ####Motivations: - We are aiming to share UBO and SSBO structures between GLSL and C++. This means we will use many of the existing vector types and others we currently don't have (uintX, intX). All these variations were asking for many more code duplication. - Deduplicate existing code which is duplicated for each vector size. - We also want to share small functions. Which means that vector functions should be static and not in the class namespace. - Reduce friction to use these types in new projects due to their incompleteness. - The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a bit of a let down. Most clases are incomplete, out of sync with each others with different codestyles, and some functions that should be static are not (i.e: `float3::reflect()`). ####Upsides: - Still support `.x, .y, .z, .w` for readability. - Compact, readable and easilly extendable. - All of the vector functions are available for all the vectors types and can be restricted to certain types. Also template specialization let us define exception for special class (like mpq). - With optimization ON, the compiler unroll the loops and performance is the same. ####Downsides: - Might impact debugability. Though I would arge that the bugs are rarelly caused by the vector class itself (since the operations are quite trivial) but by the type conversions. - Might impact compile time. I did not saw a significant impact since the usage is not really widespread. - Functions needs to be rewritten to support arbitrary vector length. For instance, one can't call `len_squared_v3v3` in `math::length_squared()` and call it a day. - Type cast does not work with the template version of the `math::` vector functions. Meaning you need to manually cast `float *` and `(float *)[3]` to `float3` for the function calls. i.e: `math::distance_squared(float3(nearest.co), positions[i]);` - Some parts might loose in readability: `float3::dot(v1.normalized(), v2.normalized())` becoming `math::dot(math::normalize(v1), math::normalize(v2))` But I propose, when appropriate, to use `using namespace blender::math;` on function local or file scope to increase readability. `dot(normalize(v1), normalize(v2))` ####Consideration: - Include back `.length()` method. It is quite handy and is more C++ oriented. - I considered the GLM library as a candidate for replacement. It felt like too much for what we need and would be difficult to extend / modify to our needs. - I used Macros to reduce code in operators declaration and potential copy paste bugs. This could reduce debugability and could be reverted. - This touches `delaunay_2d.cc` and the intersection code. I would like to know @howardt opinion on the matter. - The `noexcept` on the copy constructor of `mpq(2|3)` is being removed. But according to @JacquesLucke it is not a real problem for now. I would like to give a huge thanks to @JacquesLucke who helped during this and pushed me to reduce the duplication further. Reviewed By: brecht, sergey, JacquesLucke Differential Revision: https://developer.blender.org/D13791
2022-01-12 12:57:07 +01:00
edge_normals[i] = math::normalize(
Mesh: Move edges to a generic attribute Implements #95966, as the final step of #95965. This commit changes the storage of mesh edge vertex indices from the `MEdge` type to the generic `int2` attribute type. This follows the general design for geometry and the attribute system, where the data storage type and the usage semantics are separated. The main benefit of the change is reduced memory usage-- the requirements of storing mesh edges is reduced by 1/3. For example, this saves 8MB on a 1 million vertex grid. This also gives performance benefits to any memory-bound mesh processing algorithm that uses edges. Another benefit is that all of the edge's vertex indices are contiguous. In a few cases, it's helpful to process all of them as `Span<int>` rather than `Span<int2>`. Similarly, the type is more likely to match a generic format used by a library, or code that shouldn't know about specific Blender `Mesh` types. Various Notes: - The `.edge_verts` name is used to reflect a mapping between domains, similar to `.corner_verts`, etc. The period means that it the data shouldn't change arbitrarily by the user or procedural operations. - `edge[0]` is now used instead of `edge.v1` - Signed integers are used instead of unsigned to reduce the mixing of signed-ness, which can be error prone. - All of the previously used core mesh data types (`MVert`, `MEdge`, `MLoop`, `MPoly` are now deprecated. Only generic types are used). - The `vec2i` DNA type is used in the few C files where necessary. Pull Request: https://projects.blender.org/blender/blender/pulls/106638
2023-04-17 13:47:41 +02:00
math::interpolate(vert_normals[edge[0]], vert_normals[edge[1]], 0.5f));
BLI: refactor IndexMask for better performance and memory usage Goals of this refactor: * Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an `int64_t` for each index which is more than necessary in pretty much all practical cases currently. Using `int32_t` might still become limiting in the future in case we use this to index e.g. byte buffers larger than a few gigabytes. We also don't want to template `IndexMask`, because that would cause a split in the "ecosystem", or everything would have to be implemented twice or templated. * Allow for more multi-threading. The old `IndexMask` contains a single array. This is generally good but has the problem that it is hard to fill from multiple-threads when the final size is not known from the beginning. This is commonly the case when e.g. converting an array of bool to an index mask. Currently, this kind of code only runs on a single thread. * Allow for efficient set operations like join, intersect and difference. It should be possible to multi-thread those operations. * It should be possible to iterate over an `IndexMask` very efficiently. The most important part of that is to avoid all memory access when iterating over continuous ranges. For some core nodes (e.g. math nodes), we generate optimized code for the cases of irregular index masks and simple index ranges. To achieve these goals, a few compromises had to made: * Slicing of the mask (at specific indices) and random element access is `O(log #indices)` now, but with a low constant factor. It should be possible to split a mask into n approximately equally sized parts in `O(n)` though, making the time per split `O(1)`. * Using range-based for loops does not work well when iterating over a nested data structure like the new `IndexMask`. Therefor, `foreach_*` functions with callbacks have to be used. To avoid extra code complexity at the call site, the `foreach_*` methods support multi-threading out of the box. The new data structure splits an `IndexMask` into an arbitrary number of ordered `IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The indices within a segment are stored as `int16_t`. Each segment has an additional `int64_t` offset which allows storing arbitrary `int64_t` indices. This approach has the main benefits that segments can be processed/constructed individually on multiple threads without a serial bottleneck. Also it reduces the memory requirements significantly. For more details see comments in `BLI_index_mask.hh`. I did a few tests to verify that the data structure generally improves performance and does not cause regressions: * Our field evaluation benchmarks take about as much as before. This is to be expected because we already made sure that e.g. add node evaluation is vectorized. The important thing here is to check that changes to the way we iterate over the indices still allows for auto-vectorization. * Memory usage by a mask is about 1/4 of what it was before in the average case. That's mainly caused by the switch from `int64_t` to `int16_t` for indices. In the worst case, the memory requirements can be larger when there are many indices that are very far away. However, when they are far away from each other, that indicates that there aren't many indices in total. In common cases, memory usage can be way lower than 1/4 of before, because sub-ranges use static memory. * For some more specific numbers I benchmarked `IndexMask::from_bools` in `index_mask_from_selection` on 10.000.000 elements at various probabilities for `true` at every index: ``` Probability Old New 0 4.6 ms 0.8 ms 0.001 5.1 ms 1.3 ms 0.2 8.4 ms 1.8 ms 0.5 15.3 ms 3.0 ms 0.8 20.1 ms 3.0 ms 0.999 25.1 ms 1.7 ms 1 13.5 ms 1.1 ms ``` Pull Request: https://projects.blender.org/blender/blender/pulls/104629
2023-05-24 18:11:41 +02:00
});
return VArray<float3>::ForContainer(std::move(edge_normals));
}
case AttrDomain::Corner: {
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
if (no_corner_normals || true_normals) {
return mesh.attributes().adapt_domain(
Mesh: Add "free" custom normals Add a "dumb vector" storage option for custom normals, with the "custom_normal" attribute. Adjust the mesh normals caching to provide this attribute if it's available, and add a geometry node to store custom normals. ## Free Normals They're called "free" in the sense that they're just direction vectors in the object's local space, rather than the existing "smooth corner fan space" storage. They're also "free" in that they make further normals calculation very inexpensive, since we just use the custom normals instead. That's a big improvement from the existing custom normals storage, which usually significantly decreases viewport performance. For example, in a simple test file just storing the vertex normals on a UV sphere, using free normals gives 25 times better playback performance and 10% lower memory usage. Free normals are adjusted when applying a transformation to the entire mesh or when realizing instances, but in general they're not updated for vertex deformations. ## Set Mesh Normal Node The new geometry node allows storing free custom normals as well as the existing corner fan space normals. When free normals are chosen, free normals can be stored on vertices, faces, or face corners. Using the face corner domain is necessary to bake existing mixed sharp and smooth edges into the custom normal vectors. The node also has a mode for storing edge and mesh sharpness, meant as a "soft" replacement to the "Set Shade Smooth" node that's a bit more convenient. ## Normal Input Node The normal node outputs free custom normals mixed to whatever domain is requested. A "true normal" output that ignores custom normals and sharpness is added as well. Across Blender, custom normals are generally accessed via face and vertex normals, when "true normals" are not requested explicitly. In many cases that means they are mixed from the face corner domain. ## Future Work 1. There are many places where propagation of free normals could be improved. They should probably be normalized after mixing, and it may be useful to not just use 0 vectors for new elements. To keep the scope of this change smaller, that sort of thing generally isn't handled here. Searching `CD_NORMAL` gives a hint of where better propagation could be useful. 2. Free normals are displayed properly in edit mode, but the existing custom normal editing operators don't work with free normals yet. This will hopefully be fairly straightforward since custom normals are usually converted to `float3` for editing anyway. Edit mode changes aren't included here because they're unnecessary for the procedural custom normals use cases. 3. Most importers can probably switch to using free normals instead, or at least provide an option for it. That will give a significant import performance improvement, and an improvement of Blender's FPS for imported scenes too. Pull Request: https://projects.blender.org/blender/blender/pulls/132583
2025-04-04 19:16:51 +02:00
VArray<float3>::ForSpan(true_normals ? mesh.face_normals_true() : mesh.face_normals()),
AttrDomain::Face,
AttrDomain::Corner);
}
return VArray<float3>::ForSpan(mesh.corner_normals());
}
default:
return {};
}
}
/** \} */
} // namespace blender::bke
namespace blender::bke {
/* -------------------------------------------------------------------- */
/** \name Attribute Access
* \{ */
std::optional<AttributeAccessor> MeshComponent::attributes() const
Geometry Nodes: new geometry attribute API Currently, there are two attribute API. The first, defined in `BKE_attribute.h` is accessible from RNA and C code. The second is implemented with `GeometryComponent` and is only accessible in C++ code. The second is widely used, but only being accessible through the `GeometrySet` API makes it awkward to use, and even impossible for types that don't correspond directly to a geometry component like `CurvesGeometry`. This patch adds a new attribute API, designed to replace the `GeometryComponent` attribute API now, and to eventually replace or be the basis of the other one. The basic idea is that there is an `AttributeAccessor` class that allows code to interact with a set of attributes owned by some geometry. The accessor itself has no ownership. `AttributeAccessor` is a simple type that can be passed around by value. That makes it easy to return it from functions and to store it in containers. For const-correctness, there is also a `MutableAttributeAccessor` that allows changing individual and can add or remove attributes. Currently, `AttributeAccessor` is composed of two pointers. The first is a pointer to the owner of the attribute data. The second is a pointer to a struct with function pointers, that is similar to a virtual function table. The functions know how to access attributes on the owner. The actual attribute access for geometries is still implemented with the `AttributeProvider` pattern, which makes it easy to support different sources of attributes on a geometry and simplifies dealing with built-in attributes. There are different ways to get an attribute accessor for a geometry: * `GeometryComponent.attributes()` * `CurvesGeometry.attributes()` * `bke::mesh_attributes(const Mesh &)` * `bke::pointcloud_attributes(const PointCloud &)` All of these also have a `_for_write` variant that returns a `MutabelAttributeAccessor`. Differential Revision: https://developer.blender.org/D15280
2022-07-08 16:16:56 +02:00
{
return AttributeAccessor(mesh_, mesh_attribute_accessor_functions());
Geometry Nodes: new geometry attribute API Currently, there are two attribute API. The first, defined in `BKE_attribute.h` is accessible from RNA and C code. The second is implemented with `GeometryComponent` and is only accessible in C++ code. The second is widely used, but only being accessible through the `GeometrySet` API makes it awkward to use, and even impossible for types that don't correspond directly to a geometry component like `CurvesGeometry`. This patch adds a new attribute API, designed to replace the `GeometryComponent` attribute API now, and to eventually replace or be the basis of the other one. The basic idea is that there is an `AttributeAccessor` class that allows code to interact with a set of attributes owned by some geometry. The accessor itself has no ownership. `AttributeAccessor` is a simple type that can be passed around by value. That makes it easy to return it from functions and to store it in containers. For const-correctness, there is also a `MutableAttributeAccessor` that allows changing individual and can add or remove attributes. Currently, `AttributeAccessor` is composed of two pointers. The first is a pointer to the owner of the attribute data. The second is a pointer to a struct with function pointers, that is similar to a virtual function table. The functions know how to access attributes on the owner. The actual attribute access for geometries is still implemented with the `AttributeProvider` pattern, which makes it easy to support different sources of attributes on a geometry and simplifies dealing with built-in attributes. There are different ways to get an attribute accessor for a geometry: * `GeometryComponent.attributes()` * `CurvesGeometry.attributes()` * `bke::mesh_attributes(const Mesh &)` * `bke::pointcloud_attributes(const PointCloud &)` All of these also have a `_for_write` variant that returns a `MutabelAttributeAccessor`. Differential Revision: https://developer.blender.org/D15280
2022-07-08 16:16:56 +02:00
}
std::optional<MutableAttributeAccessor> MeshComponent::attributes_for_write()
{
Mesh *mesh = this->get_for_write();
return MutableAttributeAccessor(mesh, mesh_attribute_accessor_functions());
}
/** \} */
} // namespace blender::bke