Goals of this refactor:
* Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an
`int64_t` for each index which is more than necessary in pretty much all
practical cases currently. Using `int32_t` might still become limiting
in the future in case we use this to index e.g. byte buffers larger than
a few gigabytes. We also don't want to template `IndexMask`, because
that would cause a split in the "ecosystem", or everything would have to
be implemented twice or templated.
* Allow for more multi-threading. The old `IndexMask` contains a single
array. This is generally good but has the problem that it is hard to fill
from multiple-threads when the final size is not known from the beginning.
This is commonly the case when e.g. converting an array of bool to an
index mask. Currently, this kind of code only runs on a single thread.
* Allow for efficient set operations like join, intersect and difference.
It should be possible to multi-thread those operations.
* It should be possible to iterate over an `IndexMask` very efficiently.
The most important part of that is to avoid all memory access when iterating
over continuous ranges. For some core nodes (e.g. math nodes), we generate
optimized code for the cases of irregular index masks and simple index ranges.
To achieve these goals, a few compromises had to made:
* Slicing of the mask (at specific indices) and random element access is
`O(log #indices)` now, but with a low constant factor. It should be possible
to split a mask into n approximately equally sized parts in `O(n)` though,
making the time per split `O(1)`.
* Using range-based for loops does not work well when iterating over a nested
data structure like the new `IndexMask`. Therefor, `foreach_*` functions with
callbacks have to be used. To avoid extra code complexity at the call site,
the `foreach_*` methods support multi-threading out of the box.
The new data structure splits an `IndexMask` into an arbitrary number of ordered
`IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The
indices within a segment are stored as `int16_t`. Each segment has an additional
`int64_t` offset which allows storing arbitrary `int64_t` indices. This approach
has the main benefits that segments can be processed/constructed individually on
multiple threads without a serial bottleneck. Also it reduces the memory
requirements significantly.
For more details see comments in `BLI_index_mask.hh`.
I did a few tests to verify that the data structure generally improves
performance and does not cause regressions:
* Our field evaluation benchmarks take about as much as before. This is to be
expected because we already made sure that e.g. add node evaluation is
vectorized. The important thing here is to check that changes to the way we
iterate over the indices still allows for auto-vectorization.
* Memory usage by a mask is about 1/4 of what it was before in the average case.
That's mainly caused by the switch from `int64_t` to `int16_t` for indices.
In the worst case, the memory requirements can be larger when there are many
indices that are very far away. However, when they are far away from each other,
that indicates that there aren't many indices in total. In common cases, memory
usage can be way lower than 1/4 of before, because sub-ranges use static memory.
* For some more specific numbers I benchmarked `IndexMask::from_bools` in
`index_mask_from_selection` on 10.000.000 elements at various probabilities for
`true` at every index:
```
Probability Old New
0 4.6 ms 0.8 ms
0.001 5.1 ms 1.3 ms
0.2 8.4 ms 1.8 ms
0.5 15.3 ms 3.0 ms
0.8 20.1 ms 3.0 ms
0.999 25.1 ms 1.7 ms
1 13.5 ms 1.1 ms
```
Pull Request: https://projects.blender.org/blender/blender/pulls/104629
It was still possible to create liboverrides of Scene IDs from the
Outliner.
Scenes are currently not officially supported, RNA API allows to create
them for experimental purposes, but UI should not.
The main issue was the fact that if a Scene is overridden, it's content
will be fully invalidated when updating the liboverride at the end of
the file reading process. Since the FileData keeps a pointer to the
active view layer, it needs to be udated then.
As a side consequence, the liblinking of global data also needs to
happen before liboverrides are updated.
Embedded IDs (master collection of scene, etc.) do not exist in the Main
data-base. However, their tags should follow these from their owners. So
e.g. if a scene is in Main, its master collection should not be tagged
as no-main.
NOTE: this is somewhat also related to our ID tags sanitizing TODO task
(#88555).
Found while invesigating #107913.
- Samplerate -> Sample rate: should be two words.
- "Falloff type the feather": typo.
- JPEG, OpenJPEG and JPEG 2000 are the official spellings of the
respective projects.
- "... boundary of image(including ...": missing space.
- "Points in . direction (cannot be changed ...)":
Plural, it is a collection of multiple points. Also do not use
contraction for "cannot".
- The Bevel modifier's "Only Vertices" option was replaced by a
Vertices mode in 2.90.
- "Metaball Types": affects one object, should be singular.
- "Metaball data-block to defined blobby surfaces": typo.
- "Wire Size": this option has nothing to do with wireframes, I suppose
it's an old terminology.
- "... (for negative speed.)": remove trailing period.
- "Smooth factor effect": the prop describes a factor for an effect,
not an effect for a factor.
- "... assigned to their vertices(ensures ...": missing space.
- "... used when faces have the ObColor mode enabled": ObColor is not
used anywhere else in the UI (since Blender 2.50).
- "Effect Children": typo -> Affect.
- "Distort Min/Max": copy-pasted from another pair of properties.
- "... dismiss menu on release.(in 1/100ths of sec)": replace period
with space.
- "resolution": field names should be capitalized.
Pull Request: https://projects.blender.org/blender/blender/pulls/108227
Combine the newer less efficient C++ implementations and the older
less convenient C functions. The maps now contain one large array of
indices, split into groups by a separate array of offset indices.
Though performance of creating the maps is relatively unchanged, the
new implementation uses 4 bytes less per source element than the C
maps, and 20 bytes less than the newer C++ functions (which also
had more overhead with larger N-gons). The usage syntax is simpler
than the C functions as well.
The reduced memory usage is helpful for when these maps are cached
in the near future. It will also allow sharing the offsets between
maps for different domains like vertex to corner and vertex to face.
A simple `GroupedSpan` class is introduced to make accessing the
topology maps much simpler. It combines offset indices and a separate
span, splitting it into chunks in an efficient way.
Pull Request: https://projects.blender.org/blender/blender/pulls/107861
When the operator was executed directly (instead of being invoked as
from the UI), `stop`, `do_update` & `progress` were passed to the job
uninitialized, but `pack_islands_startjob` assigns to them (leading to
crash).
Now write protect in `pack_islands_startjob`.
Also fix memleak (missing call to `pack_islands_freejob`).
Pull Request: https://projects.blender.org/blender/blender/pulls/108185
The file is specific for the builds created by the Blender Foundation
and strictly speaking should only be used by builds created on our
release environment.
This change introduces a CMake option which is disabled by default and
which will be enabled on our buildbot.
Ref #107295
Pull Request: https://projects.blender.org/blender/blender/pulls/108191
This adds `char *simulation_bake_directory` to the nodes modifier. The path is automatically generated the first time the modifier is baked. It is _not_ automatically changed afterwards. The path is relative to the .blend file by default. For now, the path is not exposed in the UI or Python API.
This fixes issues where renaming objects/modifiers can cause the baked data to not work anymore.
Pull Request: https://projects.blender.org/blender/blender/pulls/108201
The strlen() was passed in as a size argument, in this case it
couldn't have caused an overflow because the source & destination
happened to be the same size.
Also reduce the destination buffer size as this is used for an ID name.
- Don't use the source string length +1 as the size of the destination
(USD export & IMB_exr_get_handle_name).
- Correct undersized buffer being passed into imb_exr_insert_view_name.
- Don't use the source string length +1 as the size of the destination
(USD export & IMB_exr_get_handle_name).
- Correct undersized buffer being passed into imb_exr_insert_view_name.
The rotation options are now:
* None
* Axis-Aligned (Blender 3.3 default)
- Rotate to a minimal rectangle, either vertical or horizontal.
* Cardinal (new)
- Only 90 degree rotations are allowed.
* Any
- Blender 3.6 default.
