All of these changes should be trivial, like using `string` read/write
code for strings, and a convert a few usages of 'raw data' read/write
calls to the 'struct' ones.
No behavioral changes expected here.
Advanced ID copying code can now take a `new_owner_id` ID pointer parameter,
and use it to set the relevant 'loopback' pointer to its owner ID by the
copy code itself.
Besides avoiding the need for all code copying embedded IDs to set the
loopback pointer themselves, this also means that `lib_id` copying code
itself does not need to use `IDWALK_IGNORE_MISSING_OWNER_ID` anymore.
This change is not expected to have any effect in current codebase.
This makes the read and write API functions match more closely, and adds
asserts to check that the data size is as expected.
There are still a few places remaining that use BLO_read_data_address
and similar generic functions, these should eventually be replaced as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/120994
The main motivation for this is that it's part of a fix for #113377,
where I want to propagate the edit mesh pointers through copied
meshes in modifiers and geometry nodes, instead of just setting the
edit mesh pointer at the end of the modifier stack. That would have
two main benefits:
1. We avoid the need to write to the evaluated mesh, after evaluation
which means it can be shared directly among evaluated objects.
2. When an object's mesh is completely replaced by the mesh from another
object during evaluation (with the object info node), the final edit
mesh pointer will not be "wrong", allowing us to skip index-mapped
GPU data extraction.
Beyond that, using a shared pointer just makes things more automatic.
Handling of edit mesh data is already complicated enough, this way some
of the worry and complexity can be handled by RAII.
One thing to keep in mind is that the edit mesh's BMesh is still freed
manually with `EDBM_mesh_free_data` when leaving edit mode. I figured
that was a more conservative approach for now. Maybe eventually that
could be handled automatically with RAII too.
Pull Request: https://projects.blender.org/blender/blender/pulls/120276
After some discussions and investigation over the last couple months,
it's not clear what the "wrapper type finalize" logic is necessary for.
For edit meshes and regular meshes, normals are calculated lazily when
building the draw cache.
Apart from the unnecessary complication for mesh GPU draw data
extraction, this code also causes normals to always be calculated
when turning an edit mesh wrapper into a regular mesh. However, those
normals are immediately discarded since the edit deform cache is deleted
in the next line.
Beyond the obvious simplification, the motivation for this change is to
avoid requesting write access on the evaluated mesh and cage mesh. This
works better with implicit sharing, allowing other improvements.
Pull Request: https://projects.blender.org/blender/blender/pulls/120066
Seems to work OK in basic cases, but needs more work when copying
outside of Main at least.
Note: There is no behavioral changes expected from this commit.
Note that there are at least two known usecases for this change:
* Liboverrides, as with recursive resync and proxies conversion it
often ends up creating 'virtual' linked data that does not actually
exists in the library blend files.
* Complex versionning code (`do_versions_after_setup`) when it needs
to create new IDs (currently handling linked data that way is just not
supported!).
Implements #107847.
This adds implicit sharing support for the `MemFile` undo-step. This decreases memory
usage and increases performance.
Implicit sharing allows the undo system to take (shared) ownership of some data.
Previously, the data would always be serialized and compared to the previous undo-step.
So this turns an O(n) operation into O(1) (in terms of memory usage and time).
Read/write code that wants to make use of this has to use the new `BLO_read_shared`
and `BLO_write_shared` functions respectively. Those either make use of implicit-sharing
internally or do the "full" read/write based on a passed-in function. It seems possible to
use the same API in the future to store shared data to .blend files.
Improvements:
* Much faster undo step creation in many cases by avoiding the majority data copies
and equality checks. This fixes#98574. I found undo step creation and undo step
decoding to be 2-5 times faster in some demo files from the blender website and in
some production files from the Heist project.
* Reduced memory usage when there is large data in `bmain`. For example, when
loading the same highly subdivided mesh that I used in #106228 the memory usage
is 1.03 GB now (compared to 1.62 GB in `main` currently). The main remaining copy
of the data now is done by rendering code.
* Some significant performance improvements were also measured for the new grease
pencil type (#105540).
There is one main downside of using implicit-sharing as implemented here: `MemFile`
undo steps can't be written as .blend files anymore. This has a few consequences:
* Auto-save becomes slower (up to 3x), because it can't just save the previous undo step
anymore and does a normal save instead. This has been discussed in more detail here:
https://devtalk.blender.org/t/remove-support-for-saving-memfile-undo-steps-as-blend-files-proposal/33544
It would be nice to work towards asynchronous auto-save to alleviate this problem.
Some previous work has been done to reduce the impact of this change in 41b10424c7
and f0f304e240. This has been committed separately in efb511a76d.
* Writing `quit.blend` has to do a normal file save now. So it's a bit slower too, but it's
less of a problem in practice.
* The `USE_WRITE_CRASH_BLEND` functionality does not work anymore. It doesn't seem
to be used by anyone (removed in e90f5d03c4)
There are also benefits to not writing `MemFile` from undo steps to disk. It allows us to
more safely do undo-specific optimizations without risking corrupted .blend files. This
is especially useful when we want to preserve forward compatibility in some cases.
This requires converting data before writing the .blend files, but this conversion is not
necessary for undo steps. Trying to implement this kind of optimization in the past has
often lead to bugs (e.g. 43b37fbc93).
Another new problem is that it is harder to know the size of each undo step. Currently, a
heuristic is used to approximate the memory usage, but better solutions could be found
if necessary.
Pull Request: https://projects.blender.org/blender/blender/pulls/106903
The depsgraph CoW mechanism is a bit of a misnomer. It creates an
evaluated copy for data-blocks regardless of whether the copy will
actually be written to. The point is to have physical separation between
original and evaluated data. This is in contrast to the commonly used
performance improvement of keeping a user count and copying data
implicitly when it needs to be changed. In Blender code we call this
"implicit sharing" instead. Importantly, the dependency graph has no
idea about the _actual_ CoW behavior in Blender.
Renaming this functionality in the despgraph removes some of the
confusion that comes up when talking about this, and will hopefully
make the depsgraph less confusing to understand initially too. Wording
like "the evaluated copy" (as opposed to the original data-block) has
also become common anyway.
Pull Request: https://projects.blender.org/blender/blender/pulls/118338
This data was 'hidden' away in a util in
`lib_query.cc`, which made it hard to discover and keep up-to-date.
However, as shown by e.g. #108407, critical low-level features in ID
management code, such as remapping, now rely on this information being
valid.
Also simplify `BKE_library_id_can_use_filter_id` and
`BKE_library_id_can_use_idtype` to make them more generic, relying on
IDTypeInfo to retrieve IDtype-specific info.
No behavioral changes expected here.
With this patch, materials are kept intact in simulation zones and bake nodes
without any additional user action.
This implements the design proposed in #108410 to support referencing
data-blocks (only materials for now) in the baked data. The task also describes
why this is not a trivial issue. A previous attempt was implemented in #109703
but it didn't work well-enough.
The solution is to have an explicit `name (+ library name) -> data-block`
mapping that is stored in the modifier for each bake node and simulation zone.
The `library name` is necessary for it to be unique within a .blend file. Note
that this refers to the name of the `Library` data-block and not a file path.
The baked data only contains the names of the used data-blocks. When the baked
data is loaded, the correct material data-block is looked up from the mapping.
### Automatic Mapping Generation
The most tricky aspect of this approach is to make it feel mostly automatic.
From the user point-of-view, it should just work. Therefore, we don't want the
user to have to create the mapping manually in the majority of cases. Creating
the mapping automatically is difficult because the data-blocks that should
become part of the mapping are only known during depsgraph evaluation. So we
somehow have to gather the missing data blocks during evaluation and then write
the new mappings back to the original data.
While writing back to original data is something we do in some cases already,
the situation here is different, because we are actually creating new relations
between data-blocks. This also means that we'll have to do user-counting. Since
user counts in data-blocks are *not* atomic, we can't do that from multiple
threads at the same time. Also, under some circumstances, it may be necessary to
trigger depsgraph evaluation again after the write-back because it actually
affects the result.
To solve this, a small new API is added in `DEG_depsgraph_writeback_sync.hh`. It
allows gathering tasks which write back to original data in a synchronous way
which may also require a reevaluation.
### Accessing the Mapping
A new `BakeDataBlockMap` is passed to geometry nodes evaluation by the modifier.
This map allows getting the `ID` pointer that should be used for a specific
data-block name that is stored in baked data. It's also used to gather all the
missing data mappings during evaluation.
### Weak ID References
The baked/cached geometries may have references to other data-blocks (currently
only materials, but in the future also e.g. instanced objects/collections).
However, the pointers of these data-blocks are not stable over time. That is
especially true when storing/loading the data from disk, but also just when
playing back the animation. Therefore, the used data-blocks have to referenced
in a different way at run-time.
This is solved by adding `std::unique_ptr<bake::BakeMaterialsList>` to the
run-time data of various geometry data-blocks. If the data-block is cached over
a longer period of time (such that material pointers can't be used directly), it
stores the material name (+ library name) used by each material slot. When the
geometry is used again, the material pointers are restored using these weak name
references and the `BakeDataBlockMap`.
### Manual Mapping Management
There is a new `Data-Blocks` panel in the bake settings in the node editor
sidebar that allows inspecting and modifying the data-blocks that are used when
baking. The user can change what data-block a specific name is mapped to.
Pull Request: https://projects.blender.org/blender/blender/pulls/117043
The term `PIL` stands for "platform independent library." It exists since the `Initial Revision`
commit from 2002. Nowadays, we generally just use the `BLI` (blenlib) prefix for such code
and the `PIL` prefix feels more confusing then useful. Therefore, this patch renames the
`PIL` to `BLI`.
Pull Request: https://projects.blender.org/blender/blender/pulls/117325
When there is no default color attribute yet, when creating a new color
attribute with the store named attribute node, set the default. Though
we generally try to avoid this more magical behavior, there's little
downside in this case, and it's probably what users expect.
Don't set the active attribute though, since that's UI data that generally
shouldn't be changed procedurally. It's okay if there's no active color
attribute.
Pull Request: https://projects.blender.org/blender/blender/pulls/117249
Sharing of the normals cache between copied meshes was missing from
89e3ba4e25, which under-represented the benefits of the
change. In a simple file where geometry nodes causes a re-evaluation
without changing the normals, this increased FPS from 2.6 to 14.
After the replacement of auto smooth with a modifier, sharp edges are
always used, so the "shade smooth", "shade flat", and "smooth by angle"
operators cleared the attribute. However, often users spend significant
time manually tagging edges sharp, and the operators make it too easy to
lose that data.
To keep the old behavior by default, add an option called "Keep Sharp
Edges". Though this can make the operators "ineffective" at their goal
of changing the way the meshes look, or result in redundant data stored
on the mesh, it's a much safer default, especially as users get used to
the new workflow.
Pull Request: https://projects.blender.org/blender/blender/pulls/117069
Each value is now out of the global namespace, so they can be shorter
and easier to read. Most of this commit just adds the necessary casting
and namespace specification. `enum class` can be forward declared since
it has a specified size. We will make use of that in the next commit.
Use the standard "elements_num" naming, and use the "corner" name rather
than the old "loop" name: `verts_num`, `edges_num`, and `corners_num`.
This matches the existing `faces_num` field which was already renamed.
Pull Request: https://projects.blender.org/blender/blender/pulls/116350
Make the naming consistent with the recent change from "loop" to
"corner". Avoid the need for a special type for these triangles by
conveying the semantics in the naming instead.
- `looptris` -> `corner_tris`
- `lt` -> `tri` (or `corner_tri` when there is less context)
- `looptri_index` -> `tri_index` (or `corner_tri_index`)
- `lt->tri[0]` -> `tri[0]`
- `Span<MLoopTri>` -> `Span<int3>`
- `looptri_faces` -> `tri_faces` (or `corner_tri_faces`)
If we followed the naming pattern of "corner_verts" and "edge_verts"
exactly, we'd probably use "tri_corners" instead. But that sounds much
worse and less intuitive to me.
I've found that by using standard vector types for this sort of data,
the commonalities with other areas become much clearer, and code ends
up being naturally more data oriented. Besides that, the consistency
is nice, and we get to mostly remove use of `DNA_meshdata_types.h`.
Pull Request: https://projects.blender.org/blender/blender/pulls/116238
These shouldn't be called in hot loops at all, and mostly aren't
anymore anyway. Definining them outside of a header allows removing
the `BKE_customdata.hh` include from `BKE_mesh.hh`.
This gives better asserts in debug builds through use of Span, more
safety when name convention attributes happen to have different types
or domains, and simpler code in some cases. But the main reasoning is to
avoid relying on the specifics of CustomData more to allow us to replace
it in the future.
"mesh" reads much better than "me" since "me" is a different word.
There's no reason to avoid using two more characters here. Replacing
all of these at once is better than encountering it repeatedly and
doing the same change bit by bit.
NDEBUG is part of the C standard and disables asserts. Only this will
now be used to decide if asserts are enabled.
DEBUG was a Blender specific define, that has now been removed.
_DEBUG is a Visual Studio define for builds in Debug configuration.
Blender defines this for all platforms. This is still used in a few
places in the draw code, and in external libraries Bullet and Mantaflow.
Pull Request: https://projects.blender.org/blender/blender/pulls/115774
Convert shrinkwrap data arrays to use C++ arrays and BitVector,
use references in "EditMeshData" code, and store both structs
with `std::unique_ptr` instead of a raw allocation.
