In 3.6 the names of node group sockets were using char arrays, but now
use allocated strings. The RNA system assigns nullptr to such strings
when assigning an empty string through python (UI assignment appears to
always generate a valid string). This creates issues with many STL
functions, in particular assigning nullptr to `std::string` will crash.
We have to check for valid pointers before using them in places that
don't handle nullptrs.
Pull Request: https://projects.blender.org/blender/blender/pulls/113924
In order to recreate the behavior of some builtin nodes
which have implicit inputs for ID/Index or positions, provide
a hard-coded list of default/unconnected fields for sockets.
The point is to let assets recreate the behavior of builtin
nodes in a simple way that's easy to change in the future if
this becomes a more generalized feature. A hardcoded list
makes this predictable and simple in the meantime.
When the option is set to something besides "Default Value"
it overrides the "Hide Value" option. Otherwise the default
input would conflict with the visible default value button.
This makes it possible to add #109846 as an asset in the
essentials bundle. The design is meant to be easily
extendable for shader nodes, though that isn't included
in this commit.
Pull Request: https://projects.blender.org/blender/blender/pulls/113175
`iota` is name that has no meaning, it's not an acronym or initialism.
It's usually very cryptic when I come across it. Replacing it with a
specialized function makes the code more readible.
This idea is of remapping parameters of lazy-functions is useful
not only for the repeat zone. For example, it could be used for the
for-each zone as well.
Also, moving it to a more general place indicates that there is no
repeat-zone specific stuff in it.
This makes it easier to read generated lazy-function graphs, especially when
there are zones or group nodes, because the socket identifiers on those
are some generated internal name.
Previously, the geometry nodes modifier was converting the
viewer path to a compute context at the same time as it was
setting up side effect nodes for the geometry nodes evaluation.
Now, this is changed to be a two step process. First, the viewer
path is converted to the corresponding compute context.
Afterwards, a separate function sets side effect nodes up so
that the given node in the given compute context will be evaluated.
This has three main benefits:
* More obvious separation of concerns.
* Can reuse the code that maps a viewer path element to a compute
context already.
* With gizmo nodes (#112677), it may become necessary to add side
effect nodes based on a compute context, but without having a
corresponding viewer path.
Previously, it was only possible to inspect the data from the first iteration. That
applied to both, the viewer node as well as socket inspection. Now, there is a
new `Inspection Index` setting in the zone properties. It specifies which iteration
should be used by the inspection features.
In theory we could support features like counting the index from the end, but
that can be done separately as well, as it likely requires more UI.
Pull Request: https://projects.blender.org/blender/blender/pulls/112818
The goal is to make the evaluation of repeat zones more efficient by making
use of the lazy-function evaluation system. Performance is improved in two ways:
* Unnecessary nodes are not evaluated anymore. E.g. if a repeat zone outputs two
geometries but only one of those is actually used, the other one will not be
computed anymore.
* Support evaluating different iteration indices at the same time on different threads.
It is possible that some uses of repeat zones become slower with this refactor,
especially when each iteration does very little work and there are a lot of iterations.
The old implementation was not optimized for this use case either but now there
is a bit more overhead constant overhead per iteration than before.
On the bright side, this change can result in some very significant speedups when
some computations can be skipped. See #112421 for some example setups.
There is one todo comment for adding back-links for socket usages. Properly linking
those up can result in better (shorter) life-times for anonymous attributes. Even without
that, performance is already better than before.
The implementation reuses the existing lazy-function graph system for the repeat
zone, by building a dynamically sized graph based on the number of iterations.
Building a lazy-function graph makes it possible to use the lazyness and multi-threading
features of the lazy-function graph executor. This is much easier than reimplementing
this behavior for repeat zones specifically (hence there was only single-threaded eager
execution before).
Pull Request: https://projects.blender.org/blender/blender/pulls/112421
The issue was that the `user_data` pointer in the local user data
is not necessarily valid, because a different user-data might be
passed in when a lazy-function is called a second time (it's semantically
the same user-data but, might have a different pointer).
The solution is to not keep around the dangling pointer but to pass
it in when it is actually needed. I still keep the old constructor of the
local-user-data, because more stuff might be added to it, that needs
the user-data, as was the case in the past already (before 142541c27).
The main goal of this refactor is to simplify how a geometry node group is executed.
Previously, there was duplicated logic that turned the lazy-function graph of a node
group into a single lazy-function. Now this is done only in one place and others can
just execute the lazy-function directly, without having to worry about the underlying graph.
Pull Request: https://projects.blender.org/blender/blender/pulls/112482
Goals of the refactor:
* Simplify adding (named) graph inputs and outputs.
* Add ability to refer to a graph input or output with an index.
* Get rid of the "dummy" terminology which doesn't really help.
Previously, one would add "dummy nodes" which can then serve as input
and output nodes of the graph. Now one directly adds input and outputs
using `Graph.add_input` and `Graph.add_output`. There is one interface
node that contains all inputs and another one that contains all outputs.
Being able to refer to a graph input or output with an index makes it
more efficient to implement some algorithms. E.g. one could have a
bit span for a socket that contains all the information what graph
inputs this socket depends on.
Pull Request: https://projects.blender.org/blender/blender/pulls/112474
This is a light weight solution to passing in some extra context into
a lazy-function that is invoked by the graph executor.
The new functionality is used by #112421.
This makes the code for creating a lazy-function for the body
of a zone more reusable. Currently, it is only used by the repeat
zone, but in the future the same could be used for simulations
and for-each zones.
Currently, we create a logger for every compute context that is evaluated,
even when we don't actually log anything in that context (due to other
optimizations). Now, the logger is not created eagerly anymore.
This can be especially benefitial when there are many compute contexts
that should not log anything, e.g. if there is a repeat zone with many iterations.
In an extrem case I measured a speedup for the modifier evaluation
from 24ms to 14ms.
Calling an API function after the node panels patch does not internally
tag the node tree with `NTREE_CHANGED_INTERFACE` any more, because the
node tree is not directly accessible from `bNodeTreeInterface`. Before
node panels the API functions for interfaces could tag the tree directly
for later update consideration, which now requires explicit tagging
calls.
The fix is to add a flag and mutex directly to `bNodeTreeInterface`, so
API methods can tag after updates. This mostly copies runtime data
concepts from `bNodeTree`. The `ensure_interface_cache` method is
equivalent to `ensure_topology_cache` and should be called before
accessing `interface_inputs` and similar cache data.
Pull Request: https://projects.blender.org/blender/blender/pulls/111741
This adds a new Skip input to the Simulation Output node (design task: #112082).
It is a convenience feature that makes it easy to conditionally forward the
output of the Simulation Input node to the Simulation Output node, without the
need for potentially multiple Switch nodes. When Skip is enabled, the other inputs
of the Simulation Output node are not evaluated, i.e. the nodes in the simulation
zone are ignored.
The implementation adds this new functionality directly to the `LazyFunction`
of the Simulation Output node. It has new inputs that are linked directly
to the Simulation Input node, so that the simulation state can be forwarded.
Pull Request: https://projects.blender.org/blender/blender/pulls/112140
Goals of the refactor:
* Internal support for baking individual simulation zones (not exposed in the UI yet).
* More well-defined access to simulation data in geometry nodes. Especially, it
should be more obvious where data is modified. A similar approach should also
work for the Bake node.
Previously, there were a bunch of simulation specific properties in `GeoNodesModifierData`
and then the simulation input and output nodes would have to figure out what to do with that
data. Now, there is a new `GeoNodesSimulationParams` which controls the behavior of
simulation zones. Contrary to before, different simulation zones can now be handled
independently, even if that is not really used yet. `GeoNodesSimulationParams` has to be
subclassed by a user of the geometry nodes API. The subclass controls what each simulation
input and output node does. This some of the logic that was part of the node before, into
the modifier.
The way we store simulation data is "transposed". Previously, we stored zone data per
frame, but now we store frame data per zone. This allows different zones to be more
independent. Consequently, the way the simulation cache is accessed changed. I kept
things simpler for now, avoiding many of the methods we had before, and directly
accessing the data more often which is often simple enough. This change also makes
it theoretically possible to store baked data for separate zones independently.
A downside of this is, that existing baked data can't be read anymore. We don't really
have compatibility guarantees for this format yet, so it's ok. Users will have to bake again.
The bake folder for the modifier now contains an extra subfolder for every zone.
Drawing the cached/baked frames in the timeline is less straight forward now. Currently,
it just draws the state of one of the zones, which usually is identical to that of all other
zones. This will change in the future though, and then the timeline drawing also needs
some new UI work.
Pull Request: https://projects.blender.org/blender/blender/pulls/111623
Part 3/3 of #109135, #110272
Switch to new node group interfaces and deprecate old DNA and API.
This completes support for panels in node drawing and in node group
interface declarations in particular.
The new node group interface DNA and RNA code has been added in parts
1 and 2 (#110885, #110952) but has not be enabled yet. This commit
completes the integration by
* enabling the new RNA API
* using the new API in UI
* read/write new interfaces from blend files
* add versioning for backward compatibility
* add forward-compatible writing code to reconstruct old interfaces
All places accessing node group interface declarations should now be
using the new API. A runtime cache has been added that allows simple
linear access to socket inputs and outputs even when a panel hierarchy
is used.
Old DNA has been deprecated and should only be accessed for versioning
(inputs/outputs renamed to inputs_legacy/outputs_legacy to catch
errors). Versioning code ensures both backward and forward
compatibility of existing files.
The API for old interfaces is removed. The new API is very similar but
is defined on the `ntree.interface` instead of the `ntree` directly.
Breaking change notifications and detailed instructions for migrating
will be added.
A python test has been added for the node group API functions. This
includes new functionality such as creating panels and moving items
between different levels.
This patch does not yet contain panel representations in the modifier
UI. This has been tested in a separate branch and will be added with a
later PR (#108565).
Pull Request: https://projects.blender.org/blender/blender/pulls/111348
Including <iostream> or similar headers is quite expensive, since it
also pulls in things like <locale> and so on. In many BLI headers,
iostreams are only used to implement some sort of "debug print",
or an operator<< for ostream.
Change some of the commonly used places to instead include <iosfwd>,
which is the standard way of forward-declaring iostreams related
classes, and move the actual debug-print / operator<< implementations
into .cc files.
This is not done for templated classes though (it would be possible
to provide explicit operator<< instantiations somewhere in the
source file, but that would lead to hard-to-figure-out linker error
whenever someone would add a different template type). There, where
possible, I changed from full <iostream> include to only the needed
<ostream> part.
For Span<T>, I just removed print_as_lines since it's not used by
anything. It could be moved into a .cc file using a similar approach
as above if needed.
Doing full blender build changes include counts this way:
- <iostream> 1986 -> 978
- <sstream> 2880 -> 925
It does not affect the total build time much though, mostly because
towards the end of it there's just several CPU cores finishing
compiling OpenVDB related source files.
Pull Request: https://projects.blender.org/blender/blender/pulls/111046
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Add support in the UI for the edit mode of curves, mesh, and point
cloud objects. It's possible to control for which mode sand object
types the asset is available with a dropdown in the node header.
To make this per-mode filtering possible, the static asset tree
cache is now unique per context mode.
See #101778
Pull Request: https://projects.blender.org/blender/blender/pulls/109526
This adds support for running a set of nodes repeatedly. The number
of iterations can be controlled dynamically as an input of the repeat
zone. The repeat zone can be added in via the search or from the
Add > Utilities menu.
The main use case is to replace long repetitive node chains with a more
flexible alternative. Technically, repeat zones can also be used for
many other use cases. However, due to their serial nature, performance
is very sub-optimal when they are used to solve problems that could
be processed in parallel. Better solutions for such use cases will
be worked on separately.
Repeat zones are similar to simulation zones. The major difference is
that they have no concept of time and are always evaluated entirely in
the current frame, while in simulations only a single iteration is
evaluated per frame.
Stopping the repetition early using a dynamic condition is not yet
supported. "Break" functionality can be implemented manually using
Switch nodes in the loop for now. It's likely that this functionality
will be built into the repeat zone in the future.
For now, things are kept more simple.
Remaining Todos after this first version:
* Improve socket inspection and viewer node support. Currently, only
the first iteration is taken into account for socket inspection
and the viewer.
* Make loop evaluation more lazy. Currently, the evaluation is eager,
meaning that it evaluates some nodes even though their output may not
be required.
Pull Request: https://projects.blender.org/blender/blender/pulls/109164
This PR adds a new operator to run a node group on object geometry.
Only curves sculpt mode is supported for now, to simplify the design.
A new geometry node editor context to edit operator groups is also
added. This allows changing any node group, rather than only node
groups that are part of the active modifier context.
3D viewport menus are added with any geometry node group
asset in a catalog that contains the `Operator` tag. Currently Blender
must be restarted to refresh the list of available operators.
This is only the first phase of the node group operator feature.
Many more features will be added in next steps.
See #101778
Pull Request: https://projects.blender.org/blender/blender/pulls/108947
This wasn't covered by my other tests because in usually the geometry output
that anonymous attributes are propagated to is the first output, thus the index
was 0. In this case it didn't make it difference whether the `.index_range()` call
was there are not.
This refactors how a geometry nodes node tree is converted to a lazy-function
graph. Previously, all nodes were inserted into a single graph. This was fine
because every node was evaluated at most once per node group evaluation.
However, loops (#108896) break this assumption since now nodes may be
evaluated multiple times and thus a single flat graph does not work anymore.
Now, a separate lazy-function is build for every zone which gives us much
more flexibility for what can happen in a zone. Right now, the change only
applies to simulation zones since that's the only kind of zone we have.
Technically, those zones could be inlined, but turning them into a separate
lazy-function also does not hurt and makes it possible to test this refactor
without implementing loops first. Also, having them as separate functions
might help in the future if we integrate a substep loop directly into the
simulation zone.
The most tricky part here is to just link everything up correctly, especially
with respect to deterministic anonymous attribute lifetimes. Fortunately,
correctness can be checked visually by looking at the generated graphs.
The logging/viewer system also had to be refactored a bit, because now there
can be multiple different `ComputeContext` in a single node tree. Each zone
is in a separate `ComputeContext`. To make it work, the `ViewerPath` system
now explicitly supports zones and drawing code will look up the right logger
for showing inspection data.
No functional changes are expected, except that the spreadsheet now shows
"Simulation Zone" in the context path if the viewer is in a simulation.
Move `GeometrySet` and `GeometryComponent` and subclasses
to the `blender::bke` namespace. This wasn't done earlier since
these were one of the first C++ classes used throughout Blender,
but now it is common.
Also remove the now-unnecessary C-header, since all users of
the geometry set header are now in C++.
Pull Request: https://projects.blender.org/blender/blender/pulls/109020
Previously, there were two independent algorithms for analysing how anonymous
attributes are used in a node tree: One that just computed the `aal::RelationsInNode`
for an entire node tree and one that performed a more in depth analysis to
determine how far anonymous attributes should be propagated.
As it turns out, both operations can also be done at the same time and the result
can be cached on the node tree. This reduces the amount of code and allows for
better code reuse.
This simplification is likely only an intermediate step as things will probably have
to be refactored further to support e.g. serial loops (#108896).
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
Goals of this refactor:
* Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an
`int64_t` for each index which is more than necessary in pretty much all
practical cases currently. Using `int32_t` might still become limiting
in the future in case we use this to index e.g. byte buffers larger than
a few gigabytes. We also don't want to template `IndexMask`, because
that would cause a split in the "ecosystem", or everything would have to
be implemented twice or templated.
* Allow for more multi-threading. The old `IndexMask` contains a single
array. This is generally good but has the problem that it is hard to fill
from multiple-threads when the final size is not known from the beginning.
This is commonly the case when e.g. converting an array of bool to an
index mask. Currently, this kind of code only runs on a single thread.
* Allow for efficient set operations like join, intersect and difference.
It should be possible to multi-thread those operations.
* It should be possible to iterate over an `IndexMask` very efficiently.
The most important part of that is to avoid all memory access when iterating
over continuous ranges. For some core nodes (e.g. math nodes), we generate
optimized code for the cases of irregular index masks and simple index ranges.
To achieve these goals, a few compromises had to made:
* Slicing of the mask (at specific indices) and random element access is
`O(log #indices)` now, but with a low constant factor. It should be possible
to split a mask into n approximately equally sized parts in `O(n)` though,
making the time per split `O(1)`.
* Using range-based for loops does not work well when iterating over a nested
data structure like the new `IndexMask`. Therefor, `foreach_*` functions with
callbacks have to be used. To avoid extra code complexity at the call site,
the `foreach_*` methods support multi-threading out of the box.
The new data structure splits an `IndexMask` into an arbitrary number of ordered
`IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The
indices within a segment are stored as `int16_t`. Each segment has an additional
`int64_t` offset which allows storing arbitrary `int64_t` indices. This approach
has the main benefits that segments can be processed/constructed individually on
multiple threads without a serial bottleneck. Also it reduces the memory
requirements significantly.
For more details see comments in `BLI_index_mask.hh`.
I did a few tests to verify that the data structure generally improves
performance and does not cause regressions:
* Our field evaluation benchmarks take about as much as before. This is to be
expected because we already made sure that e.g. add node evaluation is
vectorized. The important thing here is to check that changes to the way we
iterate over the indices still allows for auto-vectorization.
* Memory usage by a mask is about 1/4 of what it was before in the average case.
That's mainly caused by the switch from `int64_t` to `int16_t` for indices.
In the worst case, the memory requirements can be larger when there are many
indices that are very far away. However, when they are far away from each other,
that indicates that there aren't many indices in total. In common cases, memory
usage can be way lower than 1/4 of before, because sub-ranges use static memory.
* For some more specific numbers I benchmarked `IndexMask::from_bools` in
`index_mask_from_selection` on 10.000.000 elements at various probabilities for
`true` at every index:
```
Probability Old New
0 4.6 ms 0.8 ms
0.001 5.1 ms 1.3 ms
0.2 8.4 ms 1.8 ms
0.5 15.3 ms 3.0 ms
0.8 20.1 ms 3.0 ms
0.999 25.1 ms 1.7 ms
1 13.5 ms 1.1 ms
```
Pull Request: https://projects.blender.org/blender/blender/pulls/104629
See: https://projects.blender.org/blender/blender/issues/103343
Changes:
1. Added `BKE_node.hh` file. New file includes old one.
2. Functions moved to new file. Redundant `(void)`, `struct` are removed.
3. All cpp includes replaced from `.h` on `.hh`.
4. Everything in `BKE_node.hh` is on `blender::bke` namespace.
5. All implementation functions moved in namespace.
6. Function names (`BKE_node_*`) changed to `blender::bke::node_*`.
7. `eNodeSizePreset` now is a class, with renamed items.
Pull Request: https://projects.blender.org/blender/blender/pulls/107790
The main goal here is to reduce the number of times thread-local data has
to be looked up using e.g. `EnumerableThreadSpecific.local()`. While this
isn't a bottleneck in many cases, it is when the action performed on the local
data is very short and that happens very often (e.g. logging used sockets
during geometry nodes evaluation).
The solution is to simply pass the thread-local data as parameter to many
functions that use it, instead of looking it up in those functions which
generally is more costly.
The lazy-function graph executor now only looks up the local data if
it knows that it might be on a new thread, otherwise it uses the local data
retrieved earlier.
Alongside with `UserData` there is `LocalUserData` now. This allows users
of the lazy-function evaluation (such as geometry nodes) to have custom
thread-local data that is passed to all the lazy-functions automatically.
This is used for logging now.
