Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.
In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
(meaning that no other algorithm is both faster and more efficient).
One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.
The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.
Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.
The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Instead of handling mmap, compression etc. all directly in readfile.c, refactor
the code to use a generic FileReader.
This makes it easier to add new compression methods or similar, and allows to
reuse the logic in other places (e.g. thumbnail reading).
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Optimization of font kerning by only caching kerning values after a
pair is encountered. Also saves unscaled values so they don't have to
be rebuilt between font size changes.
See D12274 for more details and speed comparison.
Differential Revision: https://developer.blender.org/D12274
Reviewed by Campbell Barton
This adds a setting to enable data caching, and another one to set the
maximum cache size in megabytes.
When caching is enabled we load the data for the entire animation in
memory, as we already do, however, if the data exceeds the memory limit,
render is aborted.
When caching is disabled, we simply load the data for the current frame
in memory.
Ref D10197
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D11163
The multi-function network system was able to compose multiple
multi-functions into a new one and to evaluate that efficiently.
This functionality was heavily used by the particle nodes prototype
a year ago. However, since then we only used multi-functions
without the need to compose them in geometry nodes.
The upcoming "fields" in geometry nodes will need a way to
compose multi-functions again. Unfortunately, the code removed
in this commit was not ideal for this different kind of function
composition. I've been working on an alternative that will be added
separately when it becomes needed.
I've had to update all the function nodes, because their interface
depended on the multi-function network data structure a bit.
The actual multi-function implementations are still the same though.
Several pure runtime data in this ID type were not properly cleared by
write/read processes.
Note that the initial undo step (the one leading back to initial read
file state) is still forcing re-load of image, for some reasons.
Common investigation together with Jeroen Bakker (@jbakker), thanks. See
also D12242.
This was partially broken with {rBde9ea94fc6f}.
The `Frame Step` and `Second Step` snapping options were working as if
they were `Nearest Frame` and `Nearest Second` respectively in the
`Dope Sheet` and `NLA` editors.
In the `Graph Editor` the problem was more serious:
"Second Step: ... The keyframe itself moves along as though in snapping
were active at all, while its handles 'stay behind' until it reaches
the next second boundary, at which point the teleport handles to
'catch up'".
The snapping code for these modes was spread across the transform
mode code and `recalcData` of each data type. Therefore, create a
unified snapping code for these options so that all issues are fixed in
one place.
Differetial Revision: https://developer.blender.org/D12241
`td2d->loc`, `td2d->loc2d`, `td->loc` and `td->iloc` were not being
initialized as is done with the other conversion types.
This avoids problems with transform modes becoming incompatible.
This avoids problems with incompatible transform modes that could
result in a crash.
This was not really useful, and added estra useless steps in case and ID
should not actually be written.
Further more, it prevented clearing the usercount on write, which can be
cause a false positive 'chanhged' detection in undo/redo case.
This patch exposes the Cycles Alembic Procedural through the MeshSequenceCache
modifier in order to use and test it from Blender.
To enable it, one has to switch the render feature set to experimental and
activate the Procedural in the modifier. An Alembic Procedural is then
created for each CacheFile from Blender set to use the Procedural, and each
Blender object having a MeshSequenceCache modifier is added to list of objects
of the right procedural.
The procedural's parameters derive from the CacheFile's properties which are
already exposed in the UI through the modifier, although more Cycles specific
options might be added in the future.
As there is currently no cache controls and since we load all the data at the
beginning of the render session, the procedural is only available during
viewport renders at the moment. When an Alembic procedural is rendered, data
from the archive are not read on the Blender side.
If a Cycles render is not active and the CacheFile is set to use the Cycles Procedural,
bounding boxes are used to display the objects in the scene as a signal that the
objects are not processed by Blender anymore. This is standard in other DCCs.
However this does not reduce the memory usage from Blender as the Alembic data
was already loaded either during an import or during a .blend file read.
This is mostly a hack to test the Cycles Alembic procedural until we have a
better Blender side mechanism for letting renderers load their own geometry,
which will be based on import and export settings on Collections (T68933).
Ref T79174, D3089
Reviewed By: brecht, sybren
Maniphest Tasks: T79174
Differential Revision: https://developer.blender.org/D10197
The current behavior of the Alembic importer is to only create a
`MeshSequenceCache` modifier or a `Transform Cache` constraint to imported
objects if they have some animated properties.
Since static objects do not have a cache reader, when reloading files those
objects are not updated. Currently, the only way to properly reload a file
because of this is to reimport it.
This adds an option to the importer to always add a cache reader, even if
there is no animated data, to ensure that all objects coming from Alembic
archive are linked to them and updated properly upon reloads.
Reviewed by: brecht, sybren
Ref D10197.
