Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.
In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
(meaning that no other algorithm is both faster and more efficient).
One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.
The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.
Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.
The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Instead of handling mmap, compression etc. all directly in readfile.c, refactor
the code to use a generic FileReader.
This makes it easier to add new compression methods or similar, and allows to
reuse the logic in other places (e.g. thumbnail reading).
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
This was not really useful, and added estra useless steps in case and ID
should not actually be written.
Further more, it prevented clearing the usercount on write, which can be
cause a false positive 'chanhged' detection in undo/redo case.
In the `FileList` struct, rename the `AssetLibraryReference
*asset_library` field to `asset_library_ref` -- it's a description of
which asset library is used, and not the asset library itself.
This is to make space for a future `AssetLibrary *asset_library` field,
which will point to an actual asset library struct/class.
No functional changes.
Reviewed by: Severin
Differential Revision: https://developer.blender.org/D12151
This patch fixes many minor spelling mistakes, all in comments or
console output. Mostly contractions like can't, won't, don't, its/it's,
etc.
Differential Revision: https://developer.blender.org/D11663
Reviewed by Harley Acheson
This change was prompted by D6408 which moves thumbnail extraction into
a shared function that happens use these endian defines but only links
blenlib.
There is no need for these defines to be associated with globals
so move into their own header.
Previously all blend files included the path they were saved,
causing files distributed publicly to include users local paths.
This also included developers home directories for startup & userprefs
defaults & app-templates bundled with Blender.
Now recovery information is only written for auto-save & quit.blend
since this is the only time they're intended to be used.
The `MAX_LIBARRAY` define was an annoying doublon to the `INDEX_ID_MAX` enum value
now defined in `DNA_ID.h`, and it is no more useful.
And comments were somewhat outdated. Also added an explanation about
chosen order for the `INDEX_ID_<IDTYPE>` order.
One of the core design aspects of the Asset Browser is that users can "mount"
custom asset libraries via the Preferences. Currently an asset library is just
a directory with one or more .blend files in it. We could easily support a
single .blend file as asset library as well (rather than a directory). It's
just disabled currently.
Note that in earlier designs, asset libraries were called repositories.
Idea is simple: In Preferences > File Paths, you can create custom libraries,
by setting a name and selecting a path. The name is ensured to be unique. If
the name or path are empty, the Asset Browser will not show it in the list of
available asset libraries.
The library path is not checked for validity, the Asset Browser will allow
selecting invalid libraries, but show a message instead of the file list, to
help the user understand what's going on.
Of course the actual Asset Browser UI is not part of this commit, it's in one
of the following ones.
{F9497950}
Part of the first Asset Browser milestone. Check the #asset_browser_milestone_1
project milestone on developer.blender.org.
Differential Revision: https://developer.blender.org/D9722
Reviewed by: Brecht Van Lommel, Hans Goudey
Remove DNA headers, using forward declarations where possible.
Also removed duplicate header, header including it's self
and unnecessary inclusion of libc system headers from BKE header.
There should not be much user visible here (other than T73668 being addressed).
I added the writing code already for the initial implementation of workspaces,
but we decided to keep it disabled until the top-bar design is more clear. It
was never planned to keep this disabled for so long.
Fixes T73668.
This one was a bit more tricky, because the file loading is
mixed with versioning code and because collections are
embedded into scenes.
All tests that passed before, still pass.
The data layout remains exactly the same..
This change just gives all the elements in `SDNA->structs` names,
making it more comfortable to work with the data.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D8926
Allows scripters to store additional information in the marker itself instead
of using work-around approach based on marker names and such.
Differential Revision: https://developer.blender.org/D8944
This patch tries to sanitize the types of our size parameters across our read
and write code, which is currently fairly inconsistent (using `int`, `uint`,
`size_t`...), by using `size_t` everywhere. Since in Blender file themselves
we can only store chunk of size `MAX_INT`, added some asserts to ensure that
as well.
See {T79561} for details.
Differential Revision: https://developer.blender.org/D8672
