Part 3/3 of #109135, #110272
Switch to new node group interfaces and deprecate old DNA and API.
This completes support for panels in node drawing and in node group
interface declarations in particular.
The new node group interface DNA and RNA code has been added in parts
1 and 2 (#110885, #110952) but has not be enabled yet. This commit
completes the integration by
* enabling the new RNA API
* using the new API in UI
* read/write new interfaces from blend files
* add versioning for backward compatibility
* add forward-compatible writing code to reconstruct old interfaces
All places accessing node group interface declarations should now be
using the new API. A runtime cache has been added that allows simple
linear access to socket inputs and outputs even when a panel hierarchy
is used.
Old DNA has been deprecated and should only be accessed for versioning
(inputs/outputs renamed to inputs_legacy/outputs_legacy to catch
errors). Versioning code ensures both backward and forward
compatibility of existing files.
The API for old interfaces is removed. The new API is very similar but
is defined on the `ntree.interface` instead of the `ntree` directly.
Breaking change notifications and detailed instructions for migrating
will be added.
A python test has been added for the node group API functions. This
includes new functionality such as creating panels and moving items
between different levels.
This patch does not yet contain panel representations in the modifier
UI. This has been tested in a separate branch and will be added with a
later PR (#108565).
Pull Request: https://projects.blender.org/blender/blender/pulls/111348
Armature layers (the 32 little dots) and bone groups are replaced with
Bone Collections:
- Bone collections are stored on the armature, and have a name that is
unique within that armature.
- An armature can have an arbitrary number of bone collections (instead
of the fixed 32 layers).
- Bones can be assigned to zero or more bone collections.
- Bone collections have a visibility setting, just like objects in scene
collections.
- When a bone is in at least one collection, and all its collections in
are hidden, the bone is hidden. In other cases (in any visible
collection, or in no collection at all), the bone visibility is
determined by its own 'hidden' flag.
- For now, bone collections cannot be nested; they are a flat list just
like bone groups were. Nestability of bone collections is intended to
be implemented in a later 4.x release.
- Since bone collections are defined on the armature, they can be used
from both pose mode and edit mode.
Versioning converts bone groups and armature layers to new bone
collections. Layers that do not contain any bones are skipped. The old
data structures remain in DNA and are unaltered, for limited forward
compatibility. That way at least a save with Blender 4.0 will not
immediately erase the bone group and armature layers and their bone
assignments.
Shortcuts:
- M/Shift+M in pose/edit mode: move to collection (M) and add to
collection (shift+M). This works similar to the M/Shift+M menus for
objects & scene collections.
- Ctrl+G in pose mode shows a port of the old 'bone groups' menu. This
is likely to be removed in the near future, as the functionality
overlaps with the M/Shift+M menus.
This is the first commit of a series; the bone collections feature will
be improved before the Blender 4.0 release. See #108941 for more info.
Pull request: https://projects.blender.org/blender/blender/pulls/109976
Move control over the color of bones from bone groups to the bones
themselves. Instead of using bone groups (which are defined on the pose,
and thus owned by the object), the color is stored on:
- the bone (`struct Bone`, or RNA `armature.bones['bone_name'].color`)
- a possible override on the pose bone (`struct bPoseChannel`, or RNA
`ob.pose.bones['bone_name'].color`).
When the pose bone is set to its default color, the color is determined
by the armature bone. In armature edit mode, the armature bone colors
are always used, as then the pose data is unavailable.
Versioning code converts bone group colors to bone colors. If the
Armature has a single user, the group color is stored on the bones
directly. If it has multiple users, the group colors will be stored on
the pose bones instead.
The bone group color is not removed from DNA for forward compatibility,
that is, to avoid immediate dataloss when saving a 3.6 file with 4.0.
This is part of the replacement of bone groups & armature layers with
bone collections. See the design task at #108941.
Pull request: https://projects.blender.org/blender/blender/pulls/109976
Move Auto-Offset toggle from Node Editor View menu
into the Editing > Node Editor section of User Preferences,
to reflect its use as a workflow option not configured
per editor or per file.
Pull Request: https://projects.blender.org/blender/blender/pulls/111589
These checks (shape keys and ID names) cost less than 1ms in Pets
production files, so think they are OK to run systematically.
The library consistency one is way more expensive (~200ms), so keeping
it behind the G_DEBUG_IO debug option for now.
NOTE: This code remains only executed in 'unlikely' case `G_DEBUG_IO` is
enabled. Think this should be systematically done, even though it can
have a non-neglectable cost... Will submit design task first though.
Add a call to `BKE_main_namemap_validate_and_fix` in the 'general
sanity validation' `after_liblink_merged_bmain_process` function.
This will detect, fix and report invalid ID naming issues (case found in
an early Gold production file).
This was hard coded to 8, which can still result in a number that
jitters making the overall FPS difficult to measure.
The default is still 8, but this is now a preference that can be
increased for values that don't jitter as much.
The `lib_link` callback cannot always be fully replaced/removed, as in
some case it is also doing some validation checks, or data editing based
on the result of lib_linking internal ID pointers.
The callback has been renamed for that purpose, from `read_lib` to
`read_after_liblink`. It is now called after all ID pointers have been
fully lib-linked for the current ID, but still before the call to
`do_versions_after_linking`.
This change should not have any behavioral effect. Although in theory
the side-effect of this commit (to split lib linking itself, and the
validation/further processing code) into two completely separated steps
could have some effects, in practice none are expected, and tests did
not show any changes in behavior either..
Part of implementing #105134: Removal of readfile's lib_link & expand code.
The `expand` callback is 'trivial' to replace, since it is only iterating
over ID pointers and calling a callback.
The only change in behavior here is that some pointers that were not
processed previously will now be.
In practice this is not expected to have any real effect (usually
the IDs used by these pointers would have been expanded through other
usages anyway). But it may solve a few corner cases, undocumented issues
though.
Part of implementing #105134: Removal of readfile's lib_link & expand code.
This is in prevision of EEVEE panoramic projection support.
EEVEE-Next is planned to add support for these parameters.
Not having these parameters in Blender DNA will make Cycles
and EEVEE not share the same parameters and will be confusing
for the user.
We handle forward compatibility by still writing the parameters
as ID properties as previous cycles versions expect.
Since this change will break the API compatibility it is crucial
to make it for the 4.0 release.
Related Task #109639
Pull Request: https://projects.blender.org/blender/blender/pulls/111310
On a user level this view transform provides much better handling of colors in
the over-exposed areas.
With this configuration the following display devices are available, including
AgX view transform for them:
* sRGB
* Display P3
* Rec.1886
* Rec.2020
NOTE: There is no Filmic view transform available for the newly added display
devices.
AgX also brings an implementation of False Colors view transform, which replaces
Filmic-based, and is available for all display devices.
The backward compatibility is preserved. The new files will default to AgX view
transform, which makes it non-forward compatible.
More technical details is available in the original PR #106355.
Please note that the PR has been split into more incremental changes when
was landing.
Pull Request: https://projects.blender.org/blender/blender/pulls/111099
Using this flag from linked data is always a double-edge sword, in one
end some user have been relying on it to keep around data that is not
really used as ID (like e.g. text data-blocks, node trees, see
e.g. #103687, #105687). On the other end, it often causes over-keeping
of linked data reference in production files.
From now on, when an unused linked data is to be kept around, users
should create an ID property to reference it.
Implements #106321.
Pull Request: https://projects.blender.org/blender/blender/pulls/111042
Implements the paper [A Microfacet-based Hair Scattering
Model](https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.14588) by
Weizhen Huang, Matthias B. Hullin and Johannes Hanika.
### Features:
- This is a far-field model, as opposed to the previous near-field
Principled Hair BSDF model. The hair is expected to be less noisy, but
lower roughness values takes longer to render due to numerical
integration along the hair width. The hair also appears to be flat when
viewed up-close.
- The longitudinal width of the scattering lobe differs along the
azimuth, providing a higher contrast compared to the evenly spread
scattering in the near-field Principled Hair BSDF model. For a more
detailed comparison, please refer to the original paper.
- Supports elliptical cross-sections, adding more realism as human hairs
are usually elliptical. The orientation of the cross-section is aligned
with the curve normal, which can be adjusted using geometry nodes.
Default is minimal twist. During sampling, light rays that hit outside
the hair width will continue propogating as if the material is
transparent.
- There is non-physical modulation factors for the first three
lobes (Reflection, Transmission, Secondary Reflection).
### Missing:
- A good default for cross-section orientation. There was an
attempt (9039f76928) to default the orientation to align with the curve
normal in the mathematical sense, but the stability (when animated) is
unclear and it would be a hassle to generalise to all curve types. After
the model is in main, we could experiment with the geometry nodes team
to see what works the best as a default.
Co-authored-by: Lukas Stockner <lukas.stockner@freenet.de>
Pull Request: https://projects.blender.org/blender/blender/pulls/105600
The potential data already read in `instance_weights` listbase needs to
be freed, instead of just clearing the listbase.
Also move some (very old!) versioning code out of Particle's 'lib_link'
code into proper versioning code.
This fixes a few related issues in VSE code:
* 'lib_link' code was doing 'read_data' tasks (like cleaning up internal
runtime data).
* Handling of `SEQ_TYPE_SOUND_HD` deprecated type of strips was
extremely confusing (versioning done partly in 'lib_link' code, partly
in some 2.50 do_version code).
* Still using deprecated `SEQ_TYPE_SOUND_HD` strip type outside of
versioning code.
* Missing proper usage of deprecated `SEQ_TYPE_SOUND_HD` type inside of
versioning code (!).
Note that the actual conversion from `SEQ_TYPE_SOUND_HD` to
`SEQ_TYPE_SOUND_RAM` is now done in the 'after setup' versioning
process, where it is somewhat safer to add IDs. This implies that
`SEQ_TYPE_SOUND_HD` must be taken into account throughout the whole
regular versioning process (before and after liblink).
Conversion logic itself has been moved to the VSE code.
Pull Request: https://projects.blender.org/blender/blender/pulls/111135
In a few cases (IPO conversion, Proxy conversion, ...), versioning
implies creating or removing IDs, and/or needs access to the whole Main
data-base.
So far this was done ad-hoc by adding some code at the end of
`setup_app_data`.
This commit formalizes this process by adding a BLO call
(`BLO_read_do_version_after_setup`) that will encapsulate all such
complex versioning code.
NOTE: This commit does not address the existing issue that this
versioning code is never performmed when linking new data (outside of
the 'opening a blendfile' context). This topic would require its own
design task.
NOTE: This commit does not fix the few current evil cases of ID creation in
regular versioning code. This will be addressed separately.
Although this commit does modifies slightly some logic in this specific
versioning process, no behavioral changes are expected here.
Pull Request: https://projects.blender.org/blender/blender/pulls/111147
This PR adds the Lacunarity and Normalize inputs to the Noise node
similar to the Voronoi node.
The Lacunarity input controls the scale factor by which each
successive Perlin noise octave is scaled. Which was previously hard
coded to a factor of 2.
The Noise node normalizes its output to the [0, 1] range by default.
The Normalize option makes it possible for the user to disable that.
To keep the behavior consistent with past versions it is enabled by
default.
To make the aforementioned normalization control easer to implement,
the fractal noise code now accumulates signed noise and remaps the
final sum, as opposed to accumulating positive [0, 1] noise.
Pull Request: https://projects.blender.org/blender/blender/pulls/110839
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
There is no reason at all for each ID read/write callbacks to have to
deal with this generic, common data explicitely. Generic ID read/write
code is the place to handle such data (just like asset, liboverride,
etc. data are handled already).
Note behavioral change expected here.
Recent change to liboverride data model (6a86dd5f34) forgot to add the
new 'RNA item ID pointers references' to the expander code (readfile
area).
Note that this was essentially harmless, as these IDs are expected to be
referenced by actual data (or liboverride reference pointer) as well,
they should never be the only ones pointing at a linked ID. But better
to do it properly nonetheless.
Using ClangBuildAnalyzer on the whole Blender build, it was pointing
out that BLI_math.h is the heaviest "header hub" (i.e. non tiny file
that is included a lot).
However, there's very little (actually zero) source files in Blender
that need "all the math" (base, colors, vectors, matrices,
quaternions, intersection, interpolation, statistics, solvers and
time). A common use case is source files needing just vectors, or
just vectors & matrices, or just colors etc. Actually, 181 files
were including the whole math thing without needing it at all.
This change removes BLI_math.h completely, and instead in all the
places that need it, includes BLI_math_vector.h or BLI_math_color.h
and so on.
Change from that:
- BLI_math_color.h was included 1399 times -> now 408 (took 114.0sec
to parse -> now 36.3sec)
- BLI_simd.h 1403 -> 418 (109.7sec -> 34.9sec).
Full rebuild of Blender (Apple M1, Xcode, RelWithDebInfo) is not
affected much (342sec -> 334sec). Most of benefit would be when
someone's changing BLI_simd.h or BLI_math_color.h or similar files,
that now there's 3x fewer files result in a recompile.
Pull Request #110944
In RNA collections storing ID references, the name of the collection
item may not always be unique, when several IDs from different libraries
are present.
While rare, this situation can become deadly to liboverride, by causing
random but exponential liboverride hierarchies corruptions.
This has already been alleviated by using preferably both name and index
in items lookup (a05419f18b) and by reducing the risk of name collision
in general between liboverrides and their linked reference (b9becc47de).
This commit goes further, by ensuring that references to items of RNA
collections of IDs stored in liboverride operations become completely
unambiguous. This is achieved by storing an extra pointer to the item's
ID itself, when relevant.
Lookup then requires a complete match `name + ID` to be successful,
which is guaranteed to match at most a single item in the whole RNA
collection (since RNA collection of IDs do not allow duplicates, and
the ID pointer is always unique).
Note that this ID pointer is implemented as an `std::optional` one
(either directly in C++ code, or using an new liboverride operation `flag`
in DNA). This allows to smoothly transition from existing data to the
added ID pointer info (when needed), without needing any dedicated
versioning. This solution also preserves forward compatibility as much
as possible.
It may also provide marginal performances improvements in some cases, as
looking up for ID items in RNA collections will first check for the
ID pointer, which should be faster than a string comparision.
Implements #110421.
Pull Request: https://projects.blender.org/blender/blender/pulls/110773
Support name-spaced add-ons, exposed via user configurable extension
repositories.
Directories for add-ons can be added at run-time and are name-spaced to
avoid name-collisions with Python modules or add-ons from other
repositories.
This is exposed as an experimental feature "Extension Repositories".
Details:
- A `bUserExtensionRepo` type which represents a repository which is
listed in the add-ons repository.
- `JunctionModuleHandle` class to manage a package with sub-modules
which can point to arbitrary locations.
- `bpy.app.handlers._extension_repos_update_{pre/post}` internal
callbacks run before/after changes to extension repositories,
callbacks are used to sync the changes to the Python package that
exposes these to add-ons.
- The size of an add-on name has been increased so a user-defined package
prefix can be included without enforcing shorter add-on names.
- Functionality relating to package management has been left out of this
change and will be developed separately.
Further work:
- While a repository can be renamed, enabled add-ons aren't renamed.
Eventually we might want to support this although we could also
disallow renaming repositories with add-ons enabled as the name isn't
all that significant.
- Removing a repository should remove all the add-ons located in this
repository.
- Sub-module names are currently restricted to `[A-Za-z]+[A-Za-z0-9_]*`
we might want to relax this to allow unicode characters (we might
still want to disallow `-` or any characters that would prevent
attribute access in code).
Ref !110869.
Reviewed By: brecht
Remove the 'SceneCollection' structure definition from DNA, and the
compatibility code converting it to the 'modern' viewlayer system.
'SceneCollection' was part at some point of the new collection system
during 2.80 development, but was never in any published Blender release.
So this code was only ensuring compatibility with a few potential
Blender files saved from in-development builds over four years ago.
Implements #110918.
Pull Request: https://projects.blender.org/blender/blender/pulls/110926
First implementation of node previews in the shader node editor. Using
the same user interface as compositor node previews, most shader nodes
can now be previewed (except group in/output and material output).
This is currently still an experimental feature, as polishing of the
user experience and performance improvements are planned. These will
be easier to do as incremental changes on this implementation.
See #110353 for details on the work that remains to be done and known
limitations.
Implementation notes:
We take advantage of the `RenderResult` available as `ImBuf` images to
store a `Render` for every viewed nested node tree present in a
`SpaceNode`. The computation is initiated at the moment of drawing nodes
overlays.
One render is started for the current nodetree, having a `ViewLayer`
associated with each previewed node. We separate the previewed nodes in
two categories: the shader ones and the non-shader ones.
- For non-shader nodes, we use AOVs which highly speed up the rendering
process by rendering every non-shader nodes at the same time. They are
rendered in the first `ViewLayer`.
- For shader nodes, we render them each in a different `ViewLayer`, by
rerouting the node to the output of the material in the preview scene.
The preview scene takes the same aspect as the Material preview scene,
and the same preview object is used.
At the moment of drawing the node overlay, we take the `Render` of the
viewed node tree and extract the `ImBuf` of the wanted viewlayer/pass
for each previewed node.
Pull Request: https://projects.blender.org/blender/blender/pulls/110065
This patch includes set of smaller changes to address visual
inconsistencies and bugs:
- Strip previews are drawn under title bar
- All strip previews now disappear when there isn't enough space for
drawing
- Like the sound strip, the color strip expands to fill the whole strip
when there is little vertical space, "taking over" the strip title
Color is more important visual indicator than the name of the strip
- Disabling strip title no longer disables strip handle frame previews
- All strip previews are now be affected by the "Show overlays" toggle
- Turning off strip text overlay no longer makes the color strip
preview disappear
This allows irradiance volume that have less priority to
transfer lighting to the ones with higher priority.
Meaning interactive relighting from the world or
lookdev HDRI is now supported if the world isn't baked
inside the volume data.
This should improve workflow with larger scenes and
interactivity with light setups.
To help setup with dynamic objects, this patch introduce
3 new parameter to remove some components from the
irradiance grids.
Pull Request: https://projects.blender.org/blender/blender/pulls/110838
The eventual goal is to get rid of `NOD_static_types.h` because that data should
be decentralized so that nodes can be more self-contained.
The switch cases are based on the state of `NOD_static_types.h` when the
versioning code was introduced in 4638e5f99a.
Pull Request: https://projects.blender.org/blender/blender/pulls/110804
