Make `BLI_path_contains()` case-insensitive on Windows. This behaviour
is dependent on the platform Blender is running on, like the rest of
BLI_path, and not on the style of paths (Windows-style paths will be
treated case-sensitively when Blender is running on Linux/macOS).
Add function `BLI_path_contains(container, containee)` that returns true
if and only `container` contains `containee`.
Paths are normalised and converted to native path separators before
comparing. Relative paths are *not* made absolute, to simplify the
function call; if this is necessary the caller has to do this conversion
first.
Add `operator<` to C++ class to allow lexicographic ordering of UUIDs.
This will be necessary when writing asset catalogs to disk in a predictable
(i.e. ordered) manner.
Put the `bUUID` class in the `blender` namespace, instead of the
`blender::bke` namespace.
As a result, some C++ code now correctly uses the C++ class, where
previously it would use the C struct and use implicit casting where
necessary. As a result, support for initializer lists had to be
explicitly coded and in another place an explicit `::bUUID` was
necessary to avoid ambiguity.
Catalogs work like directories on disk (without hard-/symlinks), in that
an asset is only contained in one catalog.
See T90066 for design considerations.
#### Known Limitations
Only a single catalog definition file (CDF), is supported, at
`${ASSET_LIBRARY_ROOT}/blender_assets.cats.txt`. In the future this is
to be expanded to support arbitrary CDFs (like one per blend file, one
per subdirectory, etc.).
The current implementation is based on the asset browser, which in
practice means that the asset browser owns the `AssetCatalogService`
instance for the selected asset library. In the future these instances
will be accessible via a less UI-bound asset system.
The UI is still very rudimentary, only showing the catalog ID for the
currently selected asset. Most notably, the loaded catalogs are not
shown yet. The UI is being implemented and will be merged soon.
#### Catalog Identifiers
Catalogs are internally identified by UUID. In older designs this was a
human-readable name, which has the problem that it has to be kept in
sync with its semantics (so when renaming a catalog from X to Y, the
UUID can be kept the same).
Since UUIDs don't communicate any human-readable information, the
mapping from catalog UUID to its path (stored in the Catalog Definition
File, CDF) is critical for understanding which asset is stored in which
human-readable catalog. To make this less critical, and to allow manual
data reconstruction after a CDF is lost/corrupted, each catalog also has
a "simple name" that's stored along with the UUID. This is also stored
on each asset, next to the catalog UUID.
#### Writing to Disk
Before saving asset catalogs to disk, the to-be-overwritten file gets
inspected. Any new catalogs that are found thre are loaded to memory
before writing the catalogs back to disk:
- Changed catalog path: in-memory data wins
- Catalogs deleted on disk: they are recreated based on in-memory data
- Catalogs deleted in memory: deleted on disk as well
- New catalogs on disk: are loaded and thus survive the overwriting
#### Tree Design
This implements the initial tree structure to load catalogs into. See
T90608, and the basic design in T90066.
Reviewed By: Severin
Maniphest Tasks: T91552
Differential Revision: https://developer.blender.org/D12589
Make the Windows version of `BLI_dir_create_recursive()` call
`BLI_path_slash_native()` before it tries to handle the path. This
should make it possible to call it with non-native path separators.
This change was provided by our Windows platform maintainer @LazyDodo in
P2414, so I assume he agrees with this change.
Rename the `UUID` struct to `bUUID`. This avoids a symbol clash on
Windows, which also defines `UUID`. This only pops up when a `UUID`
field is used in the DNA code, which is why it wasn't a problem before
(it will be once D12589 lands).
No functional changes.
On Apple machines, call `clock_gettime()` instead of `timespec_get()`.
macOS only introduced `timespec_get()` in version 10.15 (introduced
approx two years ago, so in 2019), even though the function is from C11.
XOR the 'seconds' and 'nanoseconds' fields of the current time to seed the
RNG used for generating random UUIDs. This ensures a better seed just in
case the clock as no sub-second resolution.
Add `BLI_uuid` and `DNA_uuid_types.h` with a UUID implementation
following RFC4122 (https://datatracker.ietf.org/doc/html/rfc4122.html).
The following features are implemented:
- A struct of 128 bits that can be used in DNA definitions.
- Generation of version 4 UUIDs, that is, purely random ones.
- UUID equality function.
- String to UUID and UUID to string conversion functions that are
compatible with RFC4122.
- C++ stream operator that outputs the UUID as string.
This UUID will be used by the asset system, to uniquely identify asset
catalogs.
Reviewed By: Severin, jacqueslucke
Differential Revision: https://developer.blender.org/D12475
This adds a new `ParallelMultiFunction` which wraps another multi-function
and evaluates it with multiple threads. The speeds up field evaluation
quite a bit (the effect is most noticeable when the number of evaluations
and the field is large).
There are still other single-threaded performance bottlenecks in field
evaluation that will need to be solved separately. Most notably here
is the process of copying the computed data into the position attribute
in the Set Position node.
Differential Revision: https://developer.blender.org/D12457
Refactor of our Vfont check for font validity.
See D12068 for further details.
Differential Revision: https://developer.blender.org/D12068
Reviewed by Campbell Barton
This patch adds new Perlin noise functions to BLI. The noises are compatible
with the shading texture noises in EEVEE, SVM, and OSL.
The existing Jenkins hash functions couldn't be used because they are not
compatible with the shading implementations and an attempt at adjusting the
implementation will break compatibility in various areas of Blender. So the
simplest approach is to reimplement the relevant hashing functions inside the
noise module itself.
Additionally, this patch also adds a minimal float4 structure to use in the
interface of the noise functions.
Reviewed By: JacquesLucke
Differential Revision: https://developer.blender.org/D12443
Support extracting identifiers RNA paths into fixed size buffer
since the maximum size of the identifier is known all cases.
- Add BLI_str_unescape_ex to support limiting the destination buffer.
- Add BLI_str_quoted_substr to copy values into a fixed size buffer.
This is a similar funciton to BLI_str_quoted_substrN
that extracts the range of the quoted string
instead of allocating a new string un-escaped string.
This function was documented to return the length but returned an
error value for WIN32. While this doesn't cause any bugs at the moment,
it could cause problems in the future.
Oversight in 5496d8cd36.
Besides helping to avoid buffer overflow errors this reduces complexity
of BLI_str_utf32_as_utf8 which needed a special loop for the last 6
characters to avoid writing past the buffer bounds.
Also add BLI_str_utf8_from_unicode_len which only returns the length.
Various changes to reduce risk of out of bounds errors in utf8 seeking.
- Remove BLI_str_prev_char_utf8
This function could potentially scan past the beginning of a string.
Use BLI_str_find_prev_char_utf8 instead which takes a limiting
string start argument.
- Swap arguments for BLI_str_find_prev_char_utf8 so the stepping
argument is first and the limiting argument is last.
This matches BLI_str_find_next_char_utf8.
- Change behavior of these functions to return it the start or end
pointers instead of NULL, which complicated use of these functions
to calculate offsets.
Callers that need to check if the limits were reached can compare
the return value with the start/end pointers.
- Return 'const char *' from these functions
so they don't remove const from the input arguments.
Remove BLI_str_utf8_as_unicode_and_size and
BLI_str_utf8_as_unicode_and_size_safe.
Use BLI_str_utf8_as_unicode_step instead since it takes
a buffer bounds argument to prevent buffer over-reading.
There were multiple utf8 functions which treated
errors slightly differently.
Split BLI_str_utf8_as_unicode_step into two functions.
- BLI_str_utf8_as_unicode_step_or_error returns error value
when decoding fails and doesn't step.
- BLI_str_utf8_as_unicode_step always steps forward at least one
returning the byte value without decoding
(needed to display some latin1 file-paths).
Font drawing uses BLI_str_utf8_as_unicode_step and no longer
check for error values.
Add a string length argument to BLI_str_utf8_as_unicode_step to prevent
reading past the buffer bounds or the intended range since some callers
of this function take a string length to operate on part of the string.
Font drawing for example didn't respect the length argument,
potentially causing a buffer over-read with multi-byte characters
that could read past the end of the string.
The following command would read 5 bytes past the end of the input.
`BLF_draw(font_id, (char[]){252}, 1);`
In practice strings are typically null terminated so this didn't crash
reading past buffer bounds.
Nevertheless, this wasn't correct and could cause bugs in the future.
Clamping by the length now has the same behavior as a null byte.
Add test to ensure this is working as intended.
Changes the threshold comparison from absolute to relative.
Removes threshold for MLoopCol comparison.
Adds a compare relative threshold function.
Reviewed By: JacquesLucke
Differential Revision: https://developer.blender.org/D12273
Adds full frame implementation to these nodes operations.
When enabling "extend bounds" node option, tiled implementation
result is slightly different because it's using `TranslateOperation`
with bilinear sampling for centering.
Full frame always uses nearest to don't lose image quality.
It has the disadvantage of causing image jiggling on backdrop
when switching size values as it's not pixel perfect.
This is fixed by rounding to even.
No functional changes.
Part of T88150.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D12167
Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.
In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
(meaning that no other algorithm is both faster and more efficient).
One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.
The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.
Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.
The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Instead of handling mmap, compression etc. all directly in readfile.c, refactor
the code to use a generic FileReader.
This makes it easier to add new compression methods or similar, and allows to
reuse the logic in other places (e.g. thumbnail reading).
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
This reverts commit 41e6509818.
This broke "CubeMaskFirst" test.
Any value even slightly outside the [-1.0..1.0] range
caused the result to be nan, which can happen when calculating
the dot-product between two unit length vectors.
This patch makes us less restrictive on the allowed types of FreeType
font character maps we allow, rather than primarily unicode-only. This
allows us to use some legacy, symbol, specialty, and proprietary fonts
like Wingdings. Note we were a little less restrictive with vfonts,
used for 3D Text Objects, so this patch primarily helps VSE.
See D12124 for details and examples.
Differential Revision: https://developer.blender.org/D12124
Reviewed by Brecht Van Lommel
When displaying the names of fonts for 3D Text objects, use the same
format as shown in File Browser: Family name + Style name. They are
currently shown with Postscript Name, which doesn't match well.
see D12069 for more details.
Differential Revision: https://developer.blender.org/D12069
Reviewed by Campbell Barton
