While embedded IDs are usually considered as private local data of their
owner ID, some areas of code, like the depsgraph, can consider them as
regular IDs in some aspects.
So when swapping IDs, also properly 'counter-swap' their potential
embedded IDs, such that the pointers to the embedded IDs remain as before
swapping, even though the data of the embedded IDs is swapped.
The main target of this change is memfile undo code. There, newly read
IDs are swapped with their oldder version, so that the old address
contains the new data. This allows to avoid rebuilding some of the
depsgraph. Doing the same thing for embedded IDs should reduce even
further the needs for depsgrah rebuilds on undo steps.
This commit also gives more control over the remapping of 'self' ID
pointers inside themselves.
Pull Request: https://projects.blender.org/blender/blender/pulls/107044
Do not read IDs from types flagged as 'no undo', whether they are local
or linked.
This should have no effect currently, since all 'no undo' ID types
currently are supposedly only local data anyways (Screen, WindowManager
and WorkSpace).
These IDs kept their address, but their content has been replaced
(re-read from the memfile undo step). Add an ID tag to identify them.
As a further cleanup, systematically tag these IDs for despgraph COW,
since their data is effectively modified (though in practice all of
these IDs are expected to already have other update tags anyway).
No change in behavior is expected from this commit.
Implements #95966, as the final step of #95965.
This commit changes the storage of mesh edge vertex indices from the
`MEdge` type to the generic `int2` attribute type. This follows the
general design for geometry and the attribute system, where the data
storage type and the usage semantics are separated.
The main benefit of the change is reduced memory usage-- the
requirements of storing mesh edges is reduced by 1/3. For example,
this saves 8MB on a 1 million vertex grid. This also gives performance
benefits to any memory-bound mesh processing algorithm that uses edges.
Another benefit is that all of the edge's vertex indices are
contiguous. In a few cases, it's helpful to process all of them as
`Span<int>` rather than `Span<int2>`. Similarly, the type is more
likely to match a generic format used by a library, or code that
shouldn't know about specific Blender `Mesh` types.
Various Notes:
- The `.edge_verts` name is used to reflect a mapping between domains,
similar to `.corner_verts`, etc. The period means that it the data
shouldn't change arbitrarily by the user or procedural operations.
- `edge[0]` is now used instead of `edge.v1`
- Signed integers are used instead of unsigned to reduce the mixing
of signed-ness, which can be error prone.
- All of the previously used core mesh data types (`MVert`, `MEdge`,
`MLoop`, `MPoly` are now deprecated. Only generic types are used).
- The `vec2i` DNA type is used in the few C files where necessary.
Pull Request: https://projects.blender.org/blender/blender/pulls/106638
Re-organize ID tags in a more logical way, and keep their values
strictly increasing, splitting the free available ones in-between the
main groups (to avoid having to edit all tags values when adding a new
one).
Note that shuffling around these ID tags values should not be an issue
anymore, all of these are strictly run-time, and fully cleared in write
code when writing into a .blend file.
This also lead to the second cleanup, which is removing some asserts on
ID tag values in readcode, these are useless since the tag is cleared on
write.
Embedded IDs did not benefit from any of the recent optimizations
(especially for undo case) when writing regular IDs (cleaning up of some
pure runtime data that would generate a lot of fake 'changed on undo'
status).
Now factor out of `write_file_handle` this part of the code generating
temp ID copy with cleaned-up data for writing, and expose it in BLO API
such that IDs owning embedded ones can also use it.
This is the case e.g. of the `parent` collection pointer of collections
children of a scene's master collection, or some nodetree pointers in
the UI data (node editor).
Right now handling of this new flag is exactly the same as in owning
embedded case, the distiction between both usages will happen in future
commits.
This commit is expected to have no behavioral change at all.
This integrates the new implicit-sharing system (from fbcddfcd68)
with `CustomData`. Now the potentially long arrays referenced by custom
data layers can be shared between different systems but most importantly
between different geometries. This makes e.g. copying a mesh much cheaper
because none of the attributes has to be copied. Only when an attribute
is modified does it have to be copied.
Also see the original design task: #95845.
This reduces memory and improves performance by avoiding unnecessary
data copies. For example, the used memory after loading a highly
subdivided mesh is reduced from 2.4GB to 1.79GB. This is about 25%
less which is the expected amount because in `main` there are 4 copies
of the data:
1. The original data which is allocated when the file is loaded.
2. The copy for the depsgraph allocated during depsgraph evaluation.
3. The copy for the undo system allocated when the first undo step is
created right after loading the file.
4. GPU buffers allocated for drawing.
This patch only gets rid of copy number 2 for the depsgraph. In theory
the other copies can be removed as part of follow up PRs as well though.
-----
The patch has three main components:
* Slightly modified `CustomData` API to make it work better with implicit
sharing:
* `CD_REFERENCE` and `CD_DUPLICATE` have been removed because they are
meaningless when implicit-sharing is used.
* `CD_ASSIGN` has been removed as well because it's not an allocation
type anyway. The functionality of using existing arrays as custom
data layers has not been removed though.
* This can still be done with `CustomData_add_layer_with_data` which
also has a new argument that allows passing in information about
whether the array is shared.
* `CD_FLAG_NOFREE` has been removed because it's no longer necessary. It
only existed because of `CD_REFERENCE`.
* `CustomData_copy` and `CustomData_merge` have been split up into a
functions that do copy the actual attribute values and those that do
not. The latter functions now have the `_layout` suffix
(e.g. `CustomData_copy_layout`).
* Changes in `customdata.cc` to make it actually use implicit-sharing.
* Changes in various other files to adapt to the changes in `BKE_customdata.h`.
Pull Request: https://projects.blender.org/blender/blender/pulls/106228
This code (looking for matching new IDs to restore by name) is called
from `set_app_data` after a .blend file has been read (also in memfile
undo case).
The part handling VSE clipboard to restore ID pointers is not really
needed, since pasting VSE strips already ensures stored ID pointers are
still valid, and tries to find a new matching one otherwise.
This is mistake in the refactor of the DNA storage which unified
the camera and object storage. The bit which was missed from the
initial logic is that the autosave does not use regular file
write.
Detect this in the do-versioning code and rely on the new data
format when it exists.
A candidate for 3.5.1 release.
Pull Request: https://projects.blender.org/blender/blender/pulls/106811
Makes it possible to select multiple custom script directories in Preferences >
File Paths, replacing the single Scripts path option. Each of these directories
supports the regular script directory layout with a startup file (or files?),
add-ons, modules and presets.
When installing an add-on, the script directory can be chosen.
NOTE: Deprecates the `bpy.types.PreferencesFilePaths.script_directory`
property, and replaces `bpy.utils.script_path_pref` with
`bpy.utils.script_paths_pref`.
Pull Request: https://projects.blender.org/blender/blender/pulls/104876
fa0f295b53 exposed that there was a region missing because of some
faulty versioning.
The `fluid_motion_blur.blend` Cycles test file was crashing. Seems like
this is a pre 2.5 file that was resaved in newer versions, but the
preview region for the Sequencer was missing. Before mentioned commit,
the region would be broken when activating the preview view mode in the
Sequencer, now it would crash even on startup.
Ensure the region exists, as expected.
Introduces *ARegionType.poll()* as a way to dynamically add/remove a region. The region is still there internally, but is not accessible to the user.
Previously editors would to this manually, by either removing/adding regions altogether, or hiding them, unsetting their alignment (so no AZones are added I assume) and removing their event handlers. Polling makes this much simpler.
We plan to use this in #102879.
This patch refactors multiple editors to use region polling:
- File Browser
- Sequencer
- Clip Editor
- Preferences
Notes:
- Previously, editors would lazy-create some of the regions. Versioning is added here to ensure they are always there. Could be a separate patch.
- Some editors reuse a region in different display modes, and so additional work needs to be done to reinit regions they become available or the mode changes. Typically `V2D_IS_INIT` is unset for that, which isn't great. Could be improved, but not a new issue.
Behavior change:
- When the Preferences are opened as a regular editor, the "execution" region in the preferences that displays the *Save Preferences* button would still be there, but empty with a scrollbar.
This patch makes it disappear entirely.
## Implementation
- Introduces `ARegionType.poll()`
- Before a window is drawn, all contained regions have their poll checked, and the result is stored in a flag (`RGN_FLAG_POLL_FAILED` - runtime-only flag).
- If the result of the poll changes, the area is re-initialized and event handlers are added/removed.
- UI code checks the flag as needed.
Pull Request: https://projects.blender.org/blender/blender/pulls/105088
Implements #95967.
Currently the `MPoly` struct is 12 bytes, and stores the index of a
face's first corner and the number of corners/verts/edges. Polygons
and corners are always created in order by Blender, meaning each
face's corners will be after the previous face's corners. We can take
advantage of this fact and eliminate the redundancy in mesh face
storage by only storing a single integer corner offset for each face.
The size of the face is then encoded by the offset of the next face.
The size of a single integer is 4 bytes, so this reduces memory
usage by 3 times.
The same method is used for `CurvesGeometry`, so Blender already has
an abstraction to simplify using these offsets called `OffsetIndices`.
This class is used to easily retrieve a range of corner indices for
each face. This also gives the opportunity for sharing some logic with
curves.
Another benefit of the change is that the offsets and sizes stored in
`MPoly` can no longer disagree with each other. Storing faces in the
order of their corners can simplify some code too.
Face/polygon variables now use the `IndexRange` type, which comes with
quite a few utilities that can simplify code.
Some:
- The offset integer array has to be one longer than the face count to
avoid a branch for every face, which means the data is no longer part
of the mesh's `CustomData`.
- We lose the ability to "reference" an original mesh's offset array
until more reusable CoW from #104478 is committed. That will be added
in a separate commit.
- Since they aren't part of `CustomData`, poly offsets often have to be
copied manually.
- To simplify using `OffsetIndices` in many places, some functions and
structs in headers were moved to only compile in C++.
- All meshes created by Blender use the same order for faces and face
corners, but just in case, meshes with mismatched order are fixed by
versioning code.
- `MeshPolygon.totloop` is no longer editable in RNA. This API break is
necessary here unfortunately. It should be worth it in 3.6, since
that's the best way to allow loading meshes from 4.0, which is
important for an LTS version.
Pull Request: https://projects.blender.org/blender/blender/pulls/105938
The goal is to solve confusion of the "All rights reserved" for licensing
code under an open-source license.
The phrase "All rights reserved" comes from a historical convention that
required this phrase for the copyright protection to apply. This convention
is no longer relevant.
However, even though the phrase has no meaning in establishing the copyright
it has not lost meaning in terms of licensing.
This change makes it so code under the Blender Foundation copyright does
not use "all rights reserved". This is also how the GPL license itself
states how to apply it to the source code:
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software ...
This change does not change copyright notice in cases when the copyright
is dual (BF and an author), or just an author of the code. It also does
mot change copyright which is inherited from NaN Holding BV as it needs
some further investigation about what is the proper way to handle it.
For example
```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```
becomes
```
OIIOOutputDriver::~OIIOOutputDriver() {}
```
Saves quite some vertical space, which is especially handy for
constructors.
Pull Request: https://projects.blender.org/blender/blender/pulls/105594
Previous fix/hack a60f651502 made it so that all 'Fake User' linked
IDs are considered as directly linked, so that some IDs (like linked
Text) references are not lost on file save becaus they are not
effectively used.
However, this created unwanted references to all linked assets data,
because an asset always has its fake user set. This ended up keeping
lots of references to sub-assets when appending from the new Essentials
node groups.
NOTE: This is adding another hack to the collection, the whole handling
of 'which unused linked data reference to keep on file save' needs to be
reworked for Blender 3.6.
Add control over how bone relation lines are drawn. Instead of always
drawing from the head of the child to the tail of the parent, give users
a choice for the parent side of the line. This can now be toggled
between the head and the tail of the bone, where the tail is Blender's
original behaviour and is retained as default value.
Pull request: https://projects.blender.org/blender/blender/pulls/105427
Implements #102359.
Split the `MLoop` struct into two separate integer arrays called
`corner_verts` and `corner_edges`, referring to the vertex each corner
is attached to and the next edge around the face at each corner. These
arrays can be sliced to give access to the edges or vertices in a face.
Then they are often referred to as "poly_verts" or "poly_edges".
The main benefits are halving the necessary memory bandwidth when only
one array is used and simplifications from using regular integer indices
instead of a special-purpose struct.
The commit also starts a renaming from "loop" to "corner" in mesh code.
Like the other mesh struct of array refactors, forward compatibility is
kept by writing files with the older format. This will be done until 4.0
to ease the transition process.
Looking at a small portion of the patch should give a good impression
for the rest of the changes. I tried to make the changes as small as
possible so it's easy to tell the correctness from the diff. Though I
found Blender developers have been very inventive over the last decade
when finding different ways to loop over the corners in a face.
For performance, nearly every piece of code that deals with `Mesh` is
slightly impacted. Any algorithm that is memory bottle-necked should
see an improvement. For example, here is a comparison of interpolating
a vertex float attribute to face corners (Ryzen 3700x):
**Before** (Average: 3.7 ms, Min: 3.4 ms)
```
threading::parallel_for(loops.index_range(), 4096, [&](IndexRange range) {
for (const int64_t i : range) {
dst[i] = src[loops[i].v];
}
});
```
**After** (Average: 2.9 ms, Min: 2.6 ms)
```
array_utils::gather(src, corner_verts, dst);
```
That's an improvement of 28% to the average timings, and it's also a
simplification, since an index-based routine can be used instead.
For more examples using the new arrays, see the design task.
Pull Request: https://projects.blender.org/blender/blender/pulls/104424
133dde41bb changed how 'fake user' flag is handled with linked data.
Previous behavior was a bug/inconsistency, in that the 'directly linked'
tag would be 'over-set' and never cleared, forcing saving references to
a lot of unused linked data.
Note that ideally, 'Fake user' flag should be ignored, and the only way
to decide whether to keep or not a linked ID should be whether it's
actually used by some local data.
However, #103867 and #105687 show that this is causing issues in some cases,
where users wrongly relied on the linked data's pre-defined 'Fake user' flag
to keep their linked data in their production files, even if said data had no
real user.
While not ideal, for now we should consider 'fake user' flag for linked data
as a real usage case. A better handling of this edge-case is related to
wider designs aboud handling of 'non used' data on file save, whether
linked IDs should keep track of being explicitly or implicitly linked by
the user, etc.
These are not really about reading or writing .blend files, they are
general utilities for file-names/paths. Having to link to the
blendloader library just for these utilities is annoying.
Moved them to `BKE_blendfile.h` now, in agreement with Bastien.
Pull Request: #105825
Increase the default value for the maximum number of recently-opened
files from 10 to 20. Also change the user's preference on version bump
to this new value if their current value is exactly the old default.
Pull Request: https://projects.blender.org/blender/blender/pulls/105703
This simplifies the usage of the API and is preparation for #104478.
The `CustomData_add_layer` and `CustomData_add_layer_named` now have corresponding
`*_with_data` functions that should be used when creating the layer from existing data.
Pull Request: https://projects.blender.org/blender/blender/pulls/105708
Patch version was increased, but the file subversion should have
been increased.
Also fixes previous incorrect patch versioning code as it didn't
add all 'version code until next subversion bump goes here.' after
version bump.
Refactoring mesh code, it has become clear that local cleanups and
simplifications are limited by the need to keep a C public API for
mesh functions. This change makes code more obvious and makes further
refactoring much easier.
- Add a new `BKE_mesh.hh` header for a C++ only mesh API
- Introduce a new `blender::bke::mesh` namespace, documented here:
https://wiki.blender.org/wiki/Source/Objects/Mesh#Namespaces
- Move some functions to the new namespace, cleaning up their arguments
- Move code to `Array` and `float3` where necessary to use the new API
- Define existing inline mesh data access functions to the new header
- Keep some C API functions where necessary because of RNA
- Move all C++ files to use the new header, which includes the old one
In the future it may make sense to split up `BKE_mesh.hh` more, but for
now keeping the same name as the existing header keeps things simple.
Pull Request: https://projects.blender.org/blender/blender/pulls/105416
This is something we missed in #104599.
Without it, the default color isn't always set.
Thanks to user roman-13 for pointing out the problem,
and thanks to Hans Goudey for telling me the solution.
Pull Request: https://projects.blender.org/blender/blender/pulls/105539
Move two settings that were previously in the "View" menu of the Graph Editor into User Preferences.
It has been mentioned in the meeting by Luciano Muñoz Sessarego that it would be good to move that to the preferences so you can set it once and then forget about it.
The Settings moved are:
Only Selected Curve Keyframes
Use High Quality Display
Pull Request: https://projects.blender.org/blender/blender/pulls/104532
Currently the shade smooth status for mesh faces is stored as part of
`MPoly::flag`. As described in #95967, this moves that information
to a separate boolean attribute. It also flips its status, so the
attribute is now called `sharp_face`, which mirrors the existing
`sharp_edge` attribute. The attribute doesn't need to be allocated
when all faces are smooth. Forward compatibility is kept until
4.0 like the other mesh refactors.
This will reduce memory bandwidth requirements for some operations,
since the array of booleans uses 12 times less memory than `MPoly`.
It also allows faces to be stored more efficiently in the future, since
the flag is now unused. It's also possible to use generic functions to
process the values. For example, finding whether there is a sharp face
is just `sharp_faces.contains(true)`.
The `shade_smooth` attribute is no longer accessible with geometry nodes.
Since there were dedicated accessor nodes for that data, that shouldn't
be a problem. That's difficult to version automatically since the named
attribute nodes could be used in arbitrary combinations.
**Implementation notes:**
- The attribute and array variables in the code use the `sharp_faces`
term, to be consistent with the user-facing "sharp faces" wording,
and to avoid requiring many renames when #101689 is implemented.
- Cycles now accesses smooth face status with the generic attribute,
to avoid overhead.
- Changing the zero-value from "smooth" to "flat" takes some care to
make sure defaults are the same.
- Versioning for the edge mode extrude node is particularly complex.
New nodes are added by versioning to propagate the attribute in its
old inverted state.
- A lot of access is still done through the `CustomData` API rather
than the attribute API because of a few functions. That can be
cleaned up easily in the future.
- In the future we would benefit from a way to store attributes as a
single value for when all faces are sharp.
Pull Request: https://projects.blender.org/blender/blender/pulls/104422
Previously `SEQ_transform_single_image_check` function was used to check
if image or effect strip has only one frame of content. Now these strips
are flagged with `SEQ_SINGLE_FRAME_CONTENT`.
While previously this was implicit property of strip, benefit of having
explicit flag is that this property can be decoupled from strip length.
3e5ce23c99 introduced a regression in case the freed Main was part of a
list, and was supposed to be removed from it, since calling
`BLI_remlink` does _not_ clear the `prev`/`next` pointers of the removed
link.
This commit also contains a few more tweaks to recent related b3f42d8e98
commit.
Pull Request #105485
This commit introduces a new Main boolean flag that marks is as invalid.
Higher-level file reading code does checks on this flag to abort reading
process if needed.
This is an implementation of the #105083 design task.
Given the extense of the change, I do not think this should be
considered for 3.5 and previous LTS releases.
