Previously, it was only possible to bake to disk with geometry nodes. This patch
adds support for storing the baked data directly in the .blend file.
By default, new bakes are stored in the .blend file now. Whether a new bake
should be packed or stored on disk can be configured in two places: in the
properties of the bake node and in the bake panel of the modifier. These
settings don't affect existing bakes, only the next bake.
To unpack or pack an individual bake, there is a new operator button next to the
bake button. The icon and the label below indicate where the bake is currently
stored. The label now also contains the size of the bake.
To unpack or pack all bakes, the `File > External Data > Pack Resources / Unpack
Resources` operators can be used. The unpack operator also has a new title that
mentions the number if individual files separate from the number of bakes. This
works better than just listing a number of files because a bake can consist of
many files.
Pull Request: https://projects.blender.org/blender/blender/pulls/124230
Previously, the old bake was only overwritten file by file. This caused issues like
the one fixed in 7d77691a83. Also, it could result in invalid
bake data if some bake is canceled and the data on disk is a mix of an old and
new bake.
Sometime when baking during an animation blender could crash.
WM_set_locked_interface(job->wm, true) was inside the wm thread.
This caused a race condition between the check and the lock.
Pull Request: https://projects.blender.org/blender/blender/pulls/120258
Move the public functions from the editors/object (`ED_object.hh`)
header to the `blender::ed::object` namespace, and move all of the
implementation files to the namespace too. This provides better code
completion, makes it easier to use other C++ code, removes unnecessary
redundancy and verbosity from local uses of public functions, and more
cleanly separates different modules.
See the diff in `ED_object.hh` for the main renaming changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/119947
Small operator confirmations get separate confirm and cancel buttons,
better descriptions and configurable confirm button text. But still
popup at cursor location and can be cancelled with mouse movement.
Pull Request: https://projects.blender.org/blender/blender/pulls/118346
- "can not" -> "cannot" in many places (ambiguous, also see
Writing Style guide).
- "Bezier" -> "Bézier": proper spelling of the eponym.
- Tool keymaps: make "Uv" all caps.
- "FFMPEG" -> "FFmpeg" (official spelling)
- Use MULTIPLICATION SIGN U+00D7 instead of MULTIPLICATION X U+2715.
- "LClick" -> "LMB", "RClick" -> "RMB": this convention is used
everywhere else.
- "Save rendered the image..." -> "Save the rendered image...": typo.
- "Preserve Current retiming": title case for property.
- Bend status message: punctuation.
- "... class used to define the panel" -> "header": copy-paste error.
- "... class used to define the menu" -> "asset": copy-paste error.
- "Lights user to display objects..." -> "Lights used...": typo.
- "-setaudio require one argument" -> "requires": typo.
Some issues reported by Joan Pujolar and Tamar Mebonia.
Pull Request: https://projects.blender.org/blender/blender/pulls/117856
This adds support for baking the volume component of a geometry. Previously,
volumes were just removed in the simulation and bake node.
On disk, each volume geometry is written to a separate `.vdb` file that is stored in
the bakes `blobs` directory and referenced from the corresponding meta `.json` file.
Technically, it would also be easy to write the volume data to the same `.blob`
files that we also write e.g. mesh attributes to. However, since `.vdb` is a well
known file format, it seems reasonable to just store it as a separate file. The
serialization code doesn't really care whether it's a separate file or embedded into
a bigger file, so this decision could be made at a higher level.
Just like with other geometry types, materials are preserved. Just note that when
using the written stand-alone .vdb files, materials are not preserved.
Currently, volume grids are not deduplicated on disk. This could be added in the
future if necessary.
Pull Request: https://projects.blender.org/blender/blender/pulls/117781
With this patch, materials are kept intact in simulation zones and bake nodes
without any additional user action.
This implements the design proposed in #108410 to support referencing
data-blocks (only materials for now) in the baked data. The task also describes
why this is not a trivial issue. A previous attempt was implemented in #109703
but it didn't work well-enough.
The solution is to have an explicit `name (+ library name) -> data-block`
mapping that is stored in the modifier for each bake node and simulation zone.
The `library name` is necessary for it to be unique within a .blend file. Note
that this refers to the name of the `Library` data-block and not a file path.
The baked data only contains the names of the used data-blocks. When the baked
data is loaded, the correct material data-block is looked up from the mapping.
### Automatic Mapping Generation
The most tricky aspect of this approach is to make it feel mostly automatic.
From the user point-of-view, it should just work. Therefore, we don't want the
user to have to create the mapping manually in the majority of cases. Creating
the mapping automatically is difficult because the data-blocks that should
become part of the mapping are only known during depsgraph evaluation. So we
somehow have to gather the missing data blocks during evaluation and then write
the new mappings back to the original data.
While writing back to original data is something we do in some cases already,
the situation here is different, because we are actually creating new relations
between data-blocks. This also means that we'll have to do user-counting. Since
user counts in data-blocks are *not* atomic, we can't do that from multiple
threads at the same time. Also, under some circumstances, it may be necessary to
trigger depsgraph evaluation again after the write-back because it actually
affects the result.
To solve this, a small new API is added in `DEG_depsgraph_writeback_sync.hh`. It
allows gathering tasks which write back to original data in a synchronous way
which may also require a reevaluation.
### Accessing the Mapping
A new `BakeDataBlockMap` is passed to geometry nodes evaluation by the modifier.
This map allows getting the `ID` pointer that should be used for a specific
data-block name that is stored in baked data. It's also used to gather all the
missing data mappings during evaluation.
### Weak ID References
The baked/cached geometries may have references to other data-blocks (currently
only materials, but in the future also e.g. instanced objects/collections).
However, the pointers of these data-blocks are not stable over time. That is
especially true when storing/loading the data from disk, but also just when
playing back the animation. Therefore, the used data-blocks have to referenced
in a different way at run-time.
This is solved by adding `std::unique_ptr<bake::BakeMaterialsList>` to the
run-time data of various geometry data-blocks. If the data-block is cached over
a longer period of time (such that material pointers can't be used directly), it
stores the material name (+ library name) used by each material slot. When the
geometry is used again, the material pointers are restored using these weak name
references and the `BakeDataBlockMap`.
### Manual Mapping Management
There is a new `Data-Blocks` panel in the bake settings in the node editor
sidebar that allows inspecting and modifying the data-blocks that are used when
baking. The user can change what data-block a specific name is mapped to.
Pull Request: https://projects.blender.org/blender/blender/pulls/117043
`UUID` generally stands for "universally unique identifier". The session identifier that
we use is neither universally unique, nor does it follow the standard. Therefor, the term
"session uuid" is confusing and should be replaced.
In #116888 we briefly talked about a better name and ended up with "session uid".
The reason for "uid" instead of "id" is that the latter is a very overloaded term in Blender
already.
This patch changes all uses of "uuid" to "uid" where it's used in the context of a
"session uid". It's not always trivial to see whether a specific mention of "uuid" refers
to an actual uuid or something else. Therefore, I might have missed some renames.
I can't think of an automated way to differentiate the case.
BMesh also uses the term "uuid" sometimes in a the wrong context (e.g. `UUIDFaceStepItem`)
but there it also does not mean "session uid", so it's *not* changed by this patch.
Pull Request: https://projects.blender.org/blender/blender/pulls/117350
The term `PIL` stands for "platform independent library." It exists since the `Initial Revision`
commit from 2002. Nowadays, we generally just use the `BLI` (blenlib) prefix for such code
and the `PIL` prefix feels more confusing then useful. Therefore, this patch renames the
`PIL` to `BLI`.
Pull Request: https://projects.blender.org/blender/blender/pulls/117325
Previously, calling the bake operator was always creating a job that will
be run in parallel to Blender. This is good when baking from the UI, but may
be undesired when baking with a script.
Now, when the `exec` method of the bake operator is called, the baking
happens synchronously. That means that the operator call will return only
after the baking is done. The `invoke` method has the same behavior as
before. This approach is similar to e.g. `FLUID_OT_bake_all`.
I did notice that calling the invoke method from a script can still result
in a crash sometimes, probably due to conflicting depsgraph evaluations.
That has to be investigated more.
Except for vertex groups and a few older color types, these
are generally replaced by newer generic attribute types.
Also remove some includes of DNA_mesh_types.h, since it's
included indirectly by BKE_mesh.hh currently.
This adds a new `Bake` node which allows saving and loading intermediate geometries.
Typical use cases we want address with this currently are:
* Bake some data for use with a render engine.
* Bake parts of the node tree explicitly for better performance.
For now, the format that is written to disk is not considered to be an import/export format.
It's not guaranteed that data written with one Blender version can be read by another
Blender version. For that it's better to use proper interchange formats. Better support for
those will be added eventually as well. We also plan an `Import Bake` node that allows
reading the blender-specific baked data independent of the Bake node and at different frames.
The baking works very similar to the baking in the simulation zone (UI and implementation
wise). Major differences are:
* The Bake node has a `Bake Still` and `Bake Animation` mode.
* The Bake node doesn't do automatic caching.
Implementation details:
* Refactored how we create the Python operators for moving socket items so that it also
makes sense for non-zones.
* The `ModifierCache` stores an independent map of `SimulationNodeCache` and
`BakeNodeCache`, but both share a common data structure for the actually baked data.
* For baking, the `Bake` node is added as a side-effect-node in the modifier. This will make
sure that the node is baked even if it's currently not connected to the output.
* Had to add a new `DEG_id_tag_update_for_side_effect_request` function that is used
during baking. It's necessary because I want to evaluate the object again even though none
of its inputs changed. The reevaluation is necessary to create the baked data. Using
`DEG_id_tag_update` technically works as well, but has the problem that it also uses the
`DEG_UPDATE_SOURCE_USER_EDIT` flag which (rightly) invalidates simulation caches
which shouldn't happen here.
* Slightly refactored the timeline drawing so that it can also show the baked ranges of
Bake nodes. It does not show anything for baked nodes with a in Still mode though.
* The bake operator is refactored to bake a list of `NodeBakeRequest` which makes the
code easier to follow compared to the previous nested
`ObjectBakeData > ModifierBakeData > NodeBakeData` data structure.
* The bake operators are disabled when the .blend file is not yet saved. This is technically
only necessary when the bake path depends on the .blend file path but seems ok to force
the user anyway (otherwise the bake path may be lost as well if it's set explicitly).
* The same operators are used to bake and delete single bakes in `Bake` nodes and
`Simulation Zones`. On top of that, there are separate operators of baking and deleting all
simulation bakes (those ignore bake nodes).
* The `Bake` node remembers which inputs have been fields and thus may be baked as attributes.
For that it uses an `Is Attribute` flag on the socket item. This is needed because the baked data
may still contain attribute data, even if the inputs to the bake node are disconnected.
* Similar to simulation zones, the behavior of `Bake` nodes is passed into the geometry nodes
evaluation from the outside (from the modifier only currently). This is done by providing the
new `GeoNodesBakeParams` in `GeoNodesCallData` when executing geometry nodes.
Next Steps (mostly because they also involve simulations):
* Visualize nodes that have not been evaluated in the last evaluation.
* Fix issue with seemingly loosing baked data after undo.
* Improve error handling when baked data is not found.
* Show bake node in link drag search.
* Higher level tools for managing bakes.
Pull Request: https://projects.blender.org/blender/blender/pulls/115466
Move the three current 'status variables' (stop, update and progress)
into a single 'WorkerStatus' struct. This is cleaner and will allow for
future workin this area without having to edit tens of 'startjob'
callbacks signatures all the time.
No functional change expected here.
Note: jobs' specific internal code has been modified as little as
possible, in many cases the job's own data still just store pointers to
these three values. Ideally in the future more refactor will be using a
single pointer to the shared `wmJobWorkerStatus` data instead.
Pull Request: https://projects.blender.org/blender/blender/pulls/113343
Previously, it was only possible to bake all simulations at once. This is great
for simple use-cases that, but in more complex setups one can have independent
simulations that should also be baked independently. This patch allows baking
individual simulation zones.
Furthermore, each simulation zone can now also have its own bake path and
simulation frame range. By default the simulation frame range is the scene frame
range, but it can also be customized on the scene or simulation zone level. The
bake path is generated based on the modifier bake path by default, but can be
set to another absolute or relative (to the .blend file) path.
The timeline drawing has been modified as well to be able to show more information
in the case when some simulations are baked and others are not. Instead of showing
a line for every simulation, it shows a condensed view of the important information
using at most two lines:
Is something baked? Is something valid or invalid? Also see #112232.
Pull Request: https://projects.blender.org/blender/blender/pulls/112723
The main goal here is to rename things in a way that makes sense for
simulation baking, but also for the upcoming bake node.
This also removes some versioning code from 3.6 which initialized the
default bake path. Baked data from back then can't be loaded anymore
anyway, and the way the default path is generated is different now as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/111845
Blob stands for "binary large object" and is a known term. I used to use the term `bdata`
to mean "binary data", mainly because I didn't think of a better name. Blob is a much
better name as it captures the intend of those files much better.
This change breaks existing bakes, because I rename the folder from `bdata` to `blobs`.
I think that is ok, because I also just broke bakes two days ago in a larger refactor
(e92c59bc9b).
Pull Request: https://projects.blender.org/blender/blender/pulls/111822
The hash tables and vector blenlib headers were pulling many more
headers than they actually need, including the C base math header,
our C string API header, and the StringRef header. All of this
potentially slows down compilation and polutes autocomplete
with unrelated information.
Also remove the `ListBase` constructor for `Vector`. It wasn't used
much, and making it easy to use `ListBase` isn't worth it for the
same reasons mentioned above.
It turns out a lot of files depended on indirect includes of
`BLI_string.h` and `BLI_listbase.h`, so those are fixed here.
Pull Request: https://projects.blender.org/blender/blender/pulls/111801
`BKE_simulation_state_serialize.hh` is not necessary anymore,
because the serialization is done at a lower level.
The functionality that lived there in the past is now part of
`BKE_bake_items_serialize.hh`.
