This commit implements #125759.
It removes:
* Blender does not build on big endian systems anymore.
* Support for opening blendfiles written from a big endian system is
removed.
It keeps:
* Support to generate thumbnails from big endian blendfiles.
* BE support in `extern` or `intern` libraries, including Cycles.
* Support to open big endian versions of third party file formats:
- PLY files.
- Some image files (cineon, ...).
Pull Request: https://projects.blender.org/blender/blender/pulls/140138
This commit enables Blender 4.5 to use (to some extent) blendfiles from
Blender 5.0 and later using 'long' ID names (i.e. ID names over 63 bytes).
On a more general perspective, it also introduces safer handling of
potentially corrupted ID names in a blendfile.
This is achieved by carefully checking for non-null terminated ID names
early on in readfile process, and then:
* Truncating and ensuring uniqueness of ID names.
* Doing similar process for action slot and slot users identifiers.
* In linking (and appending) context, such IDs are totally ignored. They are
not listed, and are considered as missing if some other (valid) linked ID
attempt to indirectly link them).
* Informing users through usual reporting ways.
Technically, this mainly changes two areas of the readfile code related to IDs
themselves:
* The utils `blo_bhead_id_name` that returns the ID name of an ID BHead,
without actually reading that ID, now check for a valid null-terminated
string of `MAX_ID_NAME` max size, and returns a `nullptr` on error.
_This essentially prevents listing and linking such IDs, in any way._
* The actual ID reading code (`read_id_struct`) does the same check, and
truncate the ID name to its maximum allowed length.
* Both of above checks also set a new FileData flag
(`FD_FLAGS_HAS_LONG_ID_NAME`), which is used to ensure that ID names (and
related actions slots identifiers) remain unique, and report to info to the
user.
Implements #137608.
Branched out from !137196.
Co-authored-by: michal.krupa <michal.krupa@cdprojektred.com>
Co-authored-by: Campbell Barton <ideasman42@gmail.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/139336
The argument `-a` is used twice (render animation & animation playback).
Parsing logic works since their handled at different passes however
printing help text could return either since passes aren't used
for matching.
Workaround the problem using a deterministic lookup when printing
help text which that skips arguments that have already been handled.
If multiple threads fail at the same time (usually on the same assert), the
stack trace print can be all mixed up. This patch adds a lock to the stack trace
printing, so that they are printed on after the other.
Pull Request: https://projects.blender.org/blender/blender/pulls/139237
Correct the documentation of the `BLI_uniquename_cb()` variant that was
introduced in 7afa5aaa59 to support
dynamically-allocated strings.
Contrary to the other functions with the same name, this variant does
not actually assign the name. It returns the new name and leaves the
assignment to the caller.
Pull Request: https://projects.blender.org/blender/blender/pulls/139639
e.g. stands for "exempli gratia" in Latin which means "for example".
The best way to make sure it makes sense when writing is to just expand
it to "for example". In these cases where the text was "for e.g.", that
leaves us with "for for example" which makes no sense. This commit fixes
all 110 cases, mostly just just replacing the words with "for example",
but also restructuring the text a bit more in a few cases, mostly by
moving "e.g." to the beginning of a list in parentheses.
Pull Request: https://projects.blender.org/blender/blender/pulls/139596
Several small speedups for Voronoi node (no behavior change). This
affects Cycles and CPU execution of Voronoi node e.g. in Compositor.
- F1 mode: when evaluating distance for Voronoi cells, use a faster
distance estimation, and only do final distance calculation on the
resulting closest cell. This is only really relevant for the default
Euclidian distance, where this saves a square root per evaluated cell
(in 3D Voronoi case saves 26 square roots; in 4D case saves 80 square
roots).
- N-Sphere Radius mode: speedup by doing squared distance calculations.
We only need to find the closest one, so again doing the square root
per cell is not needed here.
Something like 5%-10% speedup for F1 3D Voronoi; more performance details
in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/139490
Error in [0] could read past the array bounds for small arrays
containing uniform values.
For tests this wasn't reporting issues with ASAN unless `malloc` was
used (instead of `MEM_malloc_arrayN`).
Resolve and add tests that run with different alignments.
[0]: f9842107aa
`IMB_load_image_from_file_descriptor` use mmap to read images, and mmap
sometimes can have erroreous state llike accessing a file from network
drives. This patch tries to handle these exceptions like `BLI_mmap_read`
on Win32 (On unix the `sigbus_handler` will be called instead).
Pull Request: https://projects.blender.org/blender/blender/pulls/139472
Add RLE encoding/decoding functions for byte arrays for the purpose
of pre-processing arrays with large spans of (mostly) uniform values
before storing them in a BArrayState.
Part of a fix for #136737.
This simplifies extending `blender::get_default_hash` to more than 4 parameters
by just adding extra factors to the list. The behavior should be the same as
before.
Pull Request: https://projects.blender.org/blender/blender/pulls/139392
- Add a custom Iterator, so it can be iterated as a 1D list.
- Add missing functions like `first`, `is_empty`, `clear`, and
subscript operator.
- Add a `size_` member variable for faster `size` calls.
- Add compile-time asserts to ensure the Capacity sizes are valid.
- Add unit tests.
See #138947 for the motivation behind this.
Pull Request: https://projects.blender.org/blender/blender/pulls/139102
Check if either the memory is zero or already matches the default value,
and copy. This simplifies a common pattern to a single line.
Preparing for default initializers in DNA (#134531).
Pull Request: https://projects.blender.org/blender/blender/pulls/138830
This patch implements basic support for evaluating function nodes on volume
grids. Conceptually, a function node always creates a new grid for the output,
though the output is often a modified version of the input. The topology of the
output grid is a union of all the input grids.
All input grids have to have the same transform. Otherwise one has to use
resampling to make grids compatible.
Non-grid inputs are allowed to be single values or fields. The fields are
evaluated in a voxel/tile context, so they compute a value per voxel or per
tile.
One optimization is missing that will probably be key in the future: the ability
to merge multiple function nodes and execute them at the same time. Currently
the entire function evaluation is started and finished for every function node
that outputs a grid. This will add significant overhead in some situations.
Implementing this optimization requires some more changes outside of the scope
of this patch though. It's good to have something that works first.
Note: Not all function nodes are supported yet, because we don't have grid types
for all of them yet. Most notably, there are no color/float4 grids yet.
Implementing those properly is not super straight forward and may require some
more changes, because there isn't a 1-to-1 mapping between grid types and socket
types (a float4 grid may correspond to a color or vector socket later on).
Using grids with function nodes and fields can result in false positive warnings
in the UI currently. That's a limitation of our current socket type inferencing
and can be improved once we have better socket shape inferencing.
Pull Request: https://projects.blender.org/blender/blender/pulls/125110
Currently, there was a lot of boilerplate to compute the compute context hash.
Now, the complexity is abstracted away to make it a simple function call.
Furthermore, this makes the compute context hash generation lazy. The goal here
is to make it very cheap to construct the compute context hash in the first,
while making it a little bit more expensive (still quite cheap overall) to
access the hash when any data has to be logged. This trade-off makes sense when
we want to pass the compute context to more lower-level places in order to be
able to create better error messages with more contextual information. For
example, we'd want to create error messages during multi-function evaluation
which happens during field evaluation within a node.
Pull Request: https://projects.blender.org/blender/blender/pulls/138912
"Origin to Geometry" was also affected
With very large meshes (report was about this failing for imported
terrain data), we lack precision in `BKE_mesh_center_median`.
First intuition was to just use doubles, but based on the work done
in !132759 to get a more numerically stable way to compute a mean,
we can use that instead.
So this PR moves `compute_sum` into `blender::array_utils`
and re-uses that for `BKE_mesh_center_median`
Pull Request: https://projects.blender.org/blender/blender/pulls/138813
Briefly about this change:
- OpenColorIO C-API is removed.
- The information about color spaces in ImBuf module is removed.
It was stored in global ListBase in colormanagement.cc.
- Both OpenColorIO and fallback implementation supports GPU drawing.
- Fallback implementation supports white point, RGB curves, etc.
- Removed check for support of GPU drawing in IMB.
Historically it was implemented in a separate library with C-API, this
is because way back C++ code needed to stay in intern. This causes all
sort of overheads, and even calls that are strictly considered bad
level.
This change moves OpenColorIO integration into a module within imbuf,
next to movie, and next to IMB_colormanagement which is the main user
of it. This allows to avoid copy of color spaces, displays, views etc
in the ImBuf: they were used to help quickly querying information to
be shown on the interface. With this change it can be stored in the
same data structures as what is used by the OpenColorIO integration.
While it might not be fully avoiding duplication it is now less, and
there is no need in the user code to maintain the copies.
In a lot of cases this change also avoids allocations done per access
to the OpenColorIO. For example, it is not needed anymore to allocate
image descriptor in a heap.
The bigger user-visible change is that the fallback implementation now
supports GLSL drawing, with the whole list of supported features, such
as curve mapping and white point. This should help simplifying code
which relies on color space conversion on GPU: there is no need to
figure out fallback solution in such cases. The only case when drawing
will not work is when there is some actual bug, or driver issue, and
shader has failed to compile.
The change avoids having an opaque type for color space, and instead
uses forward declaration. It is a bit verbose on declaration, but helps
avoiding unsafe type-casts. There are ways to solve this in the future,
like having a header for forward declaration, or to flatten the name
space a bit.
There should be no user-level changes under normal operation.
When building without OpenColorIO or the configuration has a typo or
is missing a fuller set of color management tools is applies (such as the
white point correction).
Pull Request: https://projects.blender.org/blender/blender/pulls/138433
Previously, we had shown link errors on the target node as part of the node
warning system. While better than not showing any information about invalid
links (as was the state before that), it's still not ideal because it's easy to
miss when just looking at the link.
This patch adds an error icon in the middle of the invalid link. When hovering
over it, it shows the error text. When the middle of the link is not in view but
part of the link is, then the error icon will also stay visible.
Pull Request: https://projects.blender.org/blender/blender/pulls/138529
This patch adds a new `BLI_mutex.hh` header which adds `blender::Mutex` as alias
for either `tbb::mutex` or `std::mutex` depending on whether TBB is enabled.
Description copied from the patch:
```
/**
* blender::Mutex should be used as the default mutex in Blender. It implements a subset of the API
* of std::mutex but has overall better guaranteed properties. It can be used with RAII helpers
* like std::lock_guard. However, it is not compatible with e.g. std::condition_variable. So one
* still has to use std::mutex for that case.
*
* The mutex provided by TBB has these properties:
* - It's as fast as a spin-lock in the non-contended case, i.e. when no other thread is trying to
* lock the mutex at the same time.
* - In the contended case, it spins a couple of times but then blocks to avoid draining system
* resources by spinning for a long time.
* - It's only 1 byte large, compared to e.g. 40 bytes when using the std::mutex of GCC. This makes
* it more feasible to have many smaller mutexes which can improve scalability of algorithms
* compared to using fewer larger mutexes. Also it just reduces "memory slop" across Blender.
* - It is *not* a fair mutex, i.e. it's not guaranteed that a thread will ever be able to lock the
* mutex when there are always more than one threads that try to lock it. In the majority of
* cases, using a fair mutex just causes extra overhead without any benefit. std::mutex is not
* guaranteed to be fair either.
*/
```
The performance benchmark suggests that the impact is negilible in almost
all cases. The only benchmarks that show interesting behavior are the once
testing foreach zones in Geometry Nodes. These tests are explicitly testing
overhead, which I still have to reduce over time. So it's not unexpected that
changing the mutex has an impact there. What's interesting is that on macos the
performance improves a lot while on linux it gets worse. Since that overhead
should eventually be removed almost entirely, I don't really consider that
blocking.
Links:
* Documentation of different mutex flavors in TBB:
https://www.intel.com/content/www/us/en/docs/onetbb/developer-guide-api-reference/2021-12/mutex-flavors.html
* Older implementation of a similar mutex by me:
https://archive.blender.org/developer/differential/0016/0016711/index.html
* Interesting read regarding how a mutex can be this small:
https://webkit.org/blog/6161/locking-in-webkit/
Pull Request: https://projects.blender.org/blender/blender/pulls/138370
We recently started using blenloader code in blendthumb to avoid having to
reimplement some parts of .blend file parsing. While this works, it has the side
effect that on Windows referencing blenloader code increased the binary size of
blendthumb from < 1MB to ~75 MB. That happens because this dependency drags
along lots of other code which effectively is unused, but the compiler is unable
to remove it.
There didn't seem to be a simple solution to make msvc optimize the unused code
away. This patch solves the issue by extracting the shared code into a separate
`blenloader_core` module which does not depend on the rest of Blender (except
blenlib). Therefore, using this new module in blendthumb does not drag along all
the other dependencies, bring its file size back down.
In the future, more code may be moved from blenloader to blenloader_core, but
for now I extracted these two headers:
* `BLO_core_bhead.hh`: Various `BHead` types and related parsing functions.
* `BLO_core_blend_header.hh`: Parsing of the header at the beginning of .blend
files.
Pull Request: https://projects.blender.org/blender/blender/pulls/138371
Currently, the import nodes always reimport on each evaluation. This patch adds
support for caching the loaded geometries. This is integrated with
`BLI_memory_cache.hh` and thus also takes the cache size limit into account. If
an imported file is modified on disk, the cache is invalidated. However,
Geometry Nodes will not automatically reevaluate when a file changes, so the
user would have to trigger the evaluation in some other way.
This is an alternative solution to #124369. The main benefits are that the cache
invalidation happens automatically and that the cache system is more general and
does not have to know about e.g. the different file types.
Caching speeds up node setups that heavily rely on import nodes significantly.
Pull Request: https://projects.blender.org/blender/blender/pulls/138425
The goal is to make issues like the ones fixed in #138310 become visible
much earlier in the development cycle.
This PR covers the most obvious places where it could be applied. There
are probably more places, but they can be covered later.
Pull Request: https://projects.blender.org/blender/blender/pulls/138315
There were actually two issues here:
1. The dimension reported for armatures were often wildly incorrect,
including negative values and zero!
2. The dimensions reported for objects are supposed to be invariant with
rotation, representing the dimensions along the object's local axes.
However, armature objects' reported dimensions changed with rotation.
The respective causes were:
1. `BKE_armature_min_max()` was using an incorrect formula (acknowledged
in a comment) for transforming the bounding box between spaces. This
worked fine for some of the places that `BKE_armature_min_max()` was
called, since they just reverse the transform using the same(!)
erroneous formula, but it didn't work for others.
2. `BKE_armature_min_max()` first computed the bounds in world space,
and then transformed them into object space, rather than computing
them in object space directly like the respective functions for other
object types. Even when done correctly, this causes the reported
dimension to vary with rotation.
This PR fixes these issues by simply computing the armature bounding box
in object space directly instead.
There is one place in the code base that was directly using the
world-space bounds: `view3d_calc_minmax_selected()`. However, for every
object type other than armatures, it takes the object-space bounds and
transforms them (with an incorrect formula!) to world space. So this PR
also changes `view3d_calc_minmax_selected()`'s armature code to do the
same, except with a correct formula.
Note that the reason for using the correct transform formula (departing
from other object types) is that the world-space bounds for armatures
were already correct prior to this PR due to being computed in that
space. Therefore using the incorrect formula has the potential to
introduce regressions in this case.
Pull Request: https://projects.blender.org/blender/blender/pulls/137961
Caused by a2fac05e9d.
Like the move constructor for `Vector`, all the template arguments must
be specified for the compiler to pick the move constructor over the copy
constructor when the defaults aren't used. We can test this with a non-
moveable type like unique_ptr.
Pull Request: https://projects.blender.org/blender/blender/pulls/138215
Allow configuring the inline buffer capacity for the slots array, and
add an inline buffer for the keys vector. Previously there was always
an allocation when adding an element.
The inline capacity is manually configured in a few places as part of
this commit.
Pull Request: https://projects.blender.org/blender/blender/pulls/136461
