A non UTF8 title on Wayland causes disconnection from the server.
Resolve by using the path as-is unless invalid UTF8 byte sequences
are found, in that case they're replaced by `?`.
This is skipped on WIN32 & macOS since the issue only exists on Wayland.
Similar to BLI_str_utf8_invalid_strip except that it substitutes
invalid characters and doesn't change the string length.
Useful for displaying strings that include invalid UTF8 code-points.
Change the maximum data-block name from 64 to 256 bytes by increasing MAX_ID_NAME value.
Also increase a few related non-ID data name max size, essentially the action slots identifiers, as these are the primary key used to match an Action's slot to an ID by name.
Other sub-data (bones, modifiers, etc.) lengths are not modified here, as these can be made actual dynamic strings in the future, while keeping (a reasonable level of) forward compatibility, during the course of Blender 5 release cycles.
Implements #137608.
Co-authored-by: Bastien Montagne <bastien@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/137196
The 2D->2D, 3D->3D, 4D->4D hash functions used in Voronoi node were
using quite an expensive hash function. Switch these to dedicated
2D/3D/4D hash functions (pcg2d, pcg3d, pcg4d) -- these are still very
good quality, but the hash function itself is 3x-4x faster.
Which makes Voronoi node calculation overall be around 2x faster. In
some cases when using OSL, the speedup is even larger.
This visibly changes output of the Voronoi noise however. The actual
noise "behaves" the same, just if someone was depending on the noise
pattern being exactly like it was before, this will change the pattern.
Images, more performance results and details wrt OSL are in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/139520
This commit implements #125759.
It removes:
* Blender does not build on big endian systems anymore.
* Support for opening blendfiles written from a big endian system is
removed.
It keeps:
* Support to generate thumbnails from big endian blendfiles.
* BE support in `extern` or `intern` libraries, including Cycles.
* Support to open big endian versions of third party file formats:
- PLY files.
- Some image files (cineon, ...).
Pull Request: https://projects.blender.org/blender/blender/pulls/140138
This commit enables Blender 4.5 to use (to some extent) blendfiles from
Blender 5.0 and later using 'long' ID names (i.e. ID names over 63 bytes).
On a more general perspective, it also introduces safer handling of
potentially corrupted ID names in a blendfile.
This is achieved by carefully checking for non-null terminated ID names
early on in readfile process, and then:
* Truncating and ensuring uniqueness of ID names.
* Doing similar process for action slot and slot users identifiers.
* In linking (and appending) context, such IDs are totally ignored. They are
not listed, and are considered as missing if some other (valid) linked ID
attempt to indirectly link them).
* Informing users through usual reporting ways.
Technically, this mainly changes two areas of the readfile code related to IDs
themselves:
* The utils `blo_bhead_id_name` that returns the ID name of an ID BHead,
without actually reading that ID, now check for a valid null-terminated
string of `MAX_ID_NAME` max size, and returns a `nullptr` on error.
_This essentially prevents listing and linking such IDs, in any way._
* The actual ID reading code (`read_id_struct`) does the same check, and
truncate the ID name to its maximum allowed length.
* Both of above checks also set a new FileData flag
(`FD_FLAGS_HAS_LONG_ID_NAME`), which is used to ensure that ID names (and
related actions slots identifiers) remain unique, and report to info to the
user.
Implements #137608.
Branched out from !137196.
Co-authored-by: michal.krupa <michal.krupa@cdprojektred.com>
Co-authored-by: Campbell Barton <ideasman42@gmail.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/139336
The argument `-a` is used twice (render animation & animation playback).
Parsing logic works since their handled at different passes however
printing help text could return either since passes aren't used
for matching.
Workaround the problem using a deterministic lookup when printing
help text which that skips arguments that have already been handled.
If multiple threads fail at the same time (usually on the same assert), the
stack trace print can be all mixed up. This patch adds a lock to the stack trace
printing, so that they are printed on after the other.
Pull Request: https://projects.blender.org/blender/blender/pulls/139237
e.g. stands for "exempli gratia" in Latin which means "for example".
The best way to make sure it makes sense when writing is to just expand
it to "for example". In these cases where the text was "for e.g.", that
leaves us with "for for example" which makes no sense. This commit fixes
all 110 cases, mostly just just replacing the words with "for example",
but also restructuring the text a bit more in a few cases, mostly by
moving "e.g." to the beginning of a list in parentheses.
Pull Request: https://projects.blender.org/blender/blender/pulls/139596
Several small speedups for Voronoi node (no behavior change). This
affects Cycles and CPU execution of Voronoi node e.g. in Compositor.
- F1 mode: when evaluating distance for Voronoi cells, use a faster
distance estimation, and only do final distance calculation on the
resulting closest cell. This is only really relevant for the default
Euclidian distance, where this saves a square root per evaluated cell
(in 3D Voronoi case saves 26 square roots; in 4D case saves 80 square
roots).
- N-Sphere Radius mode: speedup by doing squared distance calculations.
We only need to find the closest one, so again doing the square root
per cell is not needed here.
Something like 5%-10% speedup for F1 3D Voronoi; more performance details
in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/139490
Error in [0] could read past the array bounds for small arrays
containing uniform values.
For tests this wasn't reporting issues with ASAN unless `malloc` was
used (instead of `MEM_malloc_arrayN`).
Resolve and add tests that run with different alignments.
[0]: f9842107aa
`IMB_load_image_from_file_descriptor` use mmap to read images, and mmap
sometimes can have erroreous state llike accessing a file from network
drives. This patch tries to handle these exceptions like `BLI_mmap_read`
on Win32 (On unix the `sigbus_handler` will be called instead).
Pull Request: https://projects.blender.org/blender/blender/pulls/139472
Add RLE encoding/decoding functions for byte arrays for the purpose
of pre-processing arrays with large spans of (mostly) uniform values
before storing them in a BArrayState.
Part of a fix for #136737.
Currently, there was a lot of boilerplate to compute the compute context hash.
Now, the complexity is abstracted away to make it a simple function call.
Furthermore, this makes the compute context hash generation lazy. The goal here
is to make it very cheap to construct the compute context hash in the first,
while making it a little bit more expensive (still quite cheap overall) to
access the hash when any data has to be logged. This trade-off makes sense when
we want to pass the compute context to more lower-level places in order to be
able to create better error messages with more contextual information. For
example, we'd want to create error messages during multi-function evaluation
which happens during field evaluation within a node.
Pull Request: https://projects.blender.org/blender/blender/pulls/138912
Briefly about this change:
- OpenColorIO C-API is removed.
- The information about color spaces in ImBuf module is removed.
It was stored in global ListBase in colormanagement.cc.
- Both OpenColorIO and fallback implementation supports GPU drawing.
- Fallback implementation supports white point, RGB curves, etc.
- Removed check for support of GPU drawing in IMB.
Historically it was implemented in a separate library with C-API, this
is because way back C++ code needed to stay in intern. This causes all
sort of overheads, and even calls that are strictly considered bad
level.
This change moves OpenColorIO integration into a module within imbuf,
next to movie, and next to IMB_colormanagement which is the main user
of it. This allows to avoid copy of color spaces, displays, views etc
in the ImBuf: they were used to help quickly querying information to
be shown on the interface. With this change it can be stored in the
same data structures as what is used by the OpenColorIO integration.
While it might not be fully avoiding duplication it is now less, and
there is no need in the user code to maintain the copies.
In a lot of cases this change also avoids allocations done per access
to the OpenColorIO. For example, it is not needed anymore to allocate
image descriptor in a heap.
The bigger user-visible change is that the fallback implementation now
supports GLSL drawing, with the whole list of supported features, such
as curve mapping and white point. This should help simplifying code
which relies on color space conversion on GPU: there is no need to
figure out fallback solution in such cases. The only case when drawing
will not work is when there is some actual bug, or driver issue, and
shader has failed to compile.
The change avoids having an opaque type for color space, and instead
uses forward declaration. It is a bit verbose on declaration, but helps
avoiding unsafe type-casts. There are ways to solve this in the future,
like having a header for forward declaration, or to flatten the name
space a bit.
There should be no user-level changes under normal operation.
When building without OpenColorIO or the configuration has a typo or
is missing a fuller set of color management tools is applies (such as the
white point correction).
Pull Request: https://projects.blender.org/blender/blender/pulls/138433
Previously, we had shown link errors on the target node as part of the node
warning system. While better than not showing any information about invalid
links (as was the state before that), it's still not ideal because it's easy to
miss when just looking at the link.
This patch adds an error icon in the middle of the invalid link. When hovering
over it, it shows the error text. When the middle of the link is not in view but
part of the link is, then the error icon will also stay visible.
Pull Request: https://projects.blender.org/blender/blender/pulls/138529
This patch adds a new `BLI_mutex.hh` header which adds `blender::Mutex` as alias
for either `tbb::mutex` or `std::mutex` depending on whether TBB is enabled.
Description copied from the patch:
```
/**
* blender::Mutex should be used as the default mutex in Blender. It implements a subset of the API
* of std::mutex but has overall better guaranteed properties. It can be used with RAII helpers
* like std::lock_guard. However, it is not compatible with e.g. std::condition_variable. So one
* still has to use std::mutex for that case.
*
* The mutex provided by TBB has these properties:
* - It's as fast as a spin-lock in the non-contended case, i.e. when no other thread is trying to
* lock the mutex at the same time.
* - In the contended case, it spins a couple of times but then blocks to avoid draining system
* resources by spinning for a long time.
* - It's only 1 byte large, compared to e.g. 40 bytes when using the std::mutex of GCC. This makes
* it more feasible to have many smaller mutexes which can improve scalability of algorithms
* compared to using fewer larger mutexes. Also it just reduces "memory slop" across Blender.
* - It is *not* a fair mutex, i.e. it's not guaranteed that a thread will ever be able to lock the
* mutex when there are always more than one threads that try to lock it. In the majority of
* cases, using a fair mutex just causes extra overhead without any benefit. std::mutex is not
* guaranteed to be fair either.
*/
```
The performance benchmark suggests that the impact is negilible in almost
all cases. The only benchmarks that show interesting behavior are the once
testing foreach zones in Geometry Nodes. These tests are explicitly testing
overhead, which I still have to reduce over time. So it's not unexpected that
changing the mutex has an impact there. What's interesting is that on macos the
performance improves a lot while on linux it gets worse. Since that overhead
should eventually be removed almost entirely, I don't really consider that
blocking.
Links:
* Documentation of different mutex flavors in TBB:
https://www.intel.com/content/www/us/en/docs/onetbb/developer-guide-api-reference/2021-12/mutex-flavors.html
* Older implementation of a similar mutex by me:
https://archive.blender.org/developer/differential/0016/0016711/index.html
* Interesting read regarding how a mutex can be this small:
https://webkit.org/blog/6161/locking-in-webkit/
Pull Request: https://projects.blender.org/blender/blender/pulls/138370
We recently started using blenloader code in blendthumb to avoid having to
reimplement some parts of .blend file parsing. While this works, it has the side
effect that on Windows referencing blenloader code increased the binary size of
blendthumb from < 1MB to ~75 MB. That happens because this dependency drags
along lots of other code which effectively is unused, but the compiler is unable
to remove it.
There didn't seem to be a simple solution to make msvc optimize the unused code
away. This patch solves the issue by extracting the shared code into a separate
`blenloader_core` module which does not depend on the rest of Blender (except
blenlib). Therefore, using this new module in blendthumb does not drag along all
the other dependencies, bring its file size back down.
In the future, more code may be moved from blenloader to blenloader_core, but
for now I extracted these two headers:
* `BLO_core_bhead.hh`: Various `BHead` types and related parsing functions.
* `BLO_core_blend_header.hh`: Parsing of the header at the beginning of .blend
files.
Pull Request: https://projects.blender.org/blender/blender/pulls/138371
Currently, the import nodes always reimport on each evaluation. This patch adds
support for caching the loaded geometries. This is integrated with
`BLI_memory_cache.hh` and thus also takes the cache size limit into account. If
an imported file is modified on disk, the cache is invalidated. However,
Geometry Nodes will not automatically reevaluate when a file changes, so the
user would have to trigger the evaluation in some other way.
This is an alternative solution to #124369. The main benefits are that the cache
invalidation happens automatically and that the cache system is more general and
does not have to know about e.g. the different file types.
Caching speeds up node setups that heavily rely on import nodes significantly.
Pull Request: https://projects.blender.org/blender/blender/pulls/138425
- ImBuf reference counting: turn that into just an atomic integer
- Cachefile safety: turn into a mutex, since work under the spinlock
was quite heavy (hashtable creation, other memory allocations)
- Movie clip editor: turn into a mutex, since work under the spinlock
was very heavy (reading files from disk, etc.)
- Mesh intersect: remove the previously commented out spinlock path;
replace BLI mutex with C++ mutex for shorter code
Pull Request: https://projects.blender.org/blender/blender/pulls/137989
Previously spell checker ignored text in single quotes however this
meant incorrect spelling was ignored in text where it shouldn't have
been.
In cases single quotes were used for literal strings
(such as variables, code & compiler flags),
replace these with back-ticks.
In cases they were used for UI labels,
replace these with double quotes.
In cases they were used to reference symbols,
replace them with doxygens symbol link syntax (leading hash).
Apply some spelling corrections & tweaks (for check_spelling_* targets).
This PR adds new typographical line breaking opportunities, characters
that break depending on what follows. For example comma if not followed
by a space or a number. Period if not followed by a space or another
period. Along with related non-Latin equivalents, like Greek question
mark, full width punctuation, Hebrew, Arabic, etc.
Pull Request: https://projects.blender.org/blender/blender/pulls/137934
