Enable huge pages for jemalloc. This requests the Linux kernel to use
huge (2 MB) pages for large allocations. This has the effect of speeding
up first accesses to those allocations, and possibly also speeds up future
accesses by reducing TLB faults.
By default, 4 KB pages are used unless the user enables huge pages through
a kernel parameter or an obscure sysfs setting.
For Cycles benchmarks, this gives about a 5% CPU rendering performance
improvement. It likely also improves performance in other areas of Blender.
Pull Request: https://projects.blender.org/blender/blender/pulls/116663
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
Suppress false positive Valgrind warnings which flooded the output.
- BLI_mempool alloc/free & iteration.
- Set alignment padding bytes at the end of MEM_* allocations
as "defined" since this causes many false positive warnings
in blend file writing and MEMFILE comparisons.
- Set MEM_* allocations as undefined when `--debug-memory`
is passed in to account for debug initialization.
- Initialize pad bytes in TextLine allocations.
Recently [0] replaced back-traces from `execinfo.h` with ASAN's
`__asan_describe_address` since the linking options to hide symbols
cause the stack-traces only to include addresses (without functions).
Although using ASAN makes sense when enabled, in my tests the
stack-traces are sometimes empty. Using CMAKE_BUILD_TYPE=RelWithDebInfo
for e.g. wasn't showing a stack-trace for the leak fixed in [1].
A utility is now included to conveniently expand the addresses from
these stack traces (`tools/utils/addr2line_backtrace.py`).
Restore support for the execinfo stack-trace reporting, used when ASAN
is disabled.
[0]: 2e79ca3205
[1]: a9f0d19197
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Instead of storing the backtrace in all memory blocks, and trying to get
meaningful info out of this list of pointers when printing leaked ones,
just use `__asan_describe_address` when ASAN is enabled.
This also work on Windows, in addition to linux and (presumably) OSX,
but does require to build with ASAN enabled.
The previous code was not working very well anymore, for some reason the
call to `backtrace_symbols` seems to fail to give any meaningful
information nowadays on most of Blender code. And it was only
implemented for linux and OSX.
Based on an idea from @LazyDodo, many thanks!
Pull Request: https://projects.blender.org/blender/blender/pulls/111006
The fact that the guarded-allocated memory blocks are all linked to the
static `membase` listbase is enough for LSAN to not detect them as
leaks.
So this commit adds a new (private) callback to clear the memlist, which
is only used in the destructor of the `MemLeakDetector` class.
Many thanks to @Sergey for identifying the root issue here.
Pull Request: https://projects.blender.org/blender/blender/pulls/110995
The goal is to solve confusion of the "All rights reserved" for licensing
code under an open-source license.
The phrase "All rights reserved" comes from a historical convention that
required this phrase for the copyright protection to apply. This convention
is no longer relevant.
However, even though the phrase has no meaning in establishing the copyright
it has not lost meaning in terms of licensing.
This change makes it so code under the Blender Foundation copyright does
not use "all rights reserved". This is also how the GPL license itself
states how to apply it to the source code:
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software ...
This change does not change copyright notice in cases when the copyright
is dual (BF and an author), or just an author of the code. It also does
mot change copyright which is inherited from NaN Holding BV as it needs
some further investigation about what is the proper way to handle it.
For example
```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```
becomes
```
OIIOOutputDriver::~OIIOOutputDriver() {}
```
Saves quite some vertical space, which is especially handy for
constructors.
Pull Request: https://projects.blender.org/blender/blender/pulls/105594
The C4100 warning is related to unused formal parameters in functions.
Enabling it better aligns with "-Wunused-parameter" option in other
compilers.
While suppressing it with `__pragma(warning(suppress:4100))` is not the
same as using `__attribute__((__unused__))` in GCC or Clang, it is
still preferable to use it over completely hiding the warning.
This ensures consistent warning behavior across compilers and improves
code quality by addressing unused function parameters.
(Note that some warnings in Windows-specific code have already been
silenced in 7fcb262dfd)
Pull Request: https://projects.blender.org/blender/blender/pulls/105534
This better aligns with OSX/Linux warnings.
Although `__pragma(warning(suppress:4100))` is not the same as
`__attribute__((__unused__))` in gcc (which only affects the attribute
instead of the line), it still seems to be better to use it than to
hide the warning entirely.
This issue was introduced in rB78f28b55d39288926634d0cc.
The fix is to use a `std::shared_ptr` to ensure that the `Global` will live
long enough until all `Local` objects are destructed.
Both, the guarded and lockfree allocator, are keeping track of current
and peak memory usage. Even the lockfree allocator used to use a
global atomic variable for the memory usage. When multiple threads
use the allocator at the same time, this variable is highly contended.
This can result in significant slowdowns as presented in D16862.
While specific cases could always be optimized by reducing the number
of allocations, having this synchronization point in functions used by
almost every part of Blender is not great.
The solution is use thread-local memory counters which are only added
together when the memory usage is actually requested. For more details
see in-code comments and D16862.
Differential Revision: https://developer.blender.org/D16862
Instead of using macros like GLIBC we can use the CMake build
systems internal functions to check if some header or functions are
present on the running system's libc.
Add ./build_files/cmake/have_features.cmake to add checks for
platform features which can be used to set defines for source
files that require them.
Reviewed By: campbellbarton
Ref D15696
We have plenty of sorta generic functions, that allocate memory with
some generic name for debugging. When such a function is called and the
memory leaks, it may be unclear which call to it allocated the unfreed
memory (and thus which execution path leads to the leak).
The added function is only available if `NDEBUG` is not defined.
Differential Revision: https://developer.blender.org/D15605
Reviewed by: Sergey Sharybin, Bastien Montagne
Simplify logic for freeing a NULL pointer. While no null-pointer
de-reference was performed, this wasn't as so obvious as the pointer
was passed to MEM_lockfree_allocN_len before checking for NULL.
NOTE: T99744 claimed the a NULL pointer free was a vulnerability,
while I can't see evidence for this - exiting early makes it clearer
the memory isn't accessed.
*Details*
- Add MEMHEAD_LEN macro, avoids redundant NULL check.
- Use "UNLIKELY(..)" hint's for error cases
(freeing NULL pointer and checking if `leak_detector_has_run`).
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
- Nest compositor pages under the compositor module
- Nest GUI, DNA/RNA & externformats modules under Blender.
- Remove modules from intern which no longer exist.
- Add intern modules (atomic, eigen, glew-mx, libc_compat, locale,
numaapi, rigidbody, sky, utfconv).
- Use 'intern_' prefix for intern modules since some of the modules
use generic terms such as locale & atomic.
All platforms support the proper format string and it's already used in
various other places.
Differential Revision: https://developer.blender.org/D11460
Instead of submitting tons of tiny IO syscalls, we can speed things up
significantly by `mmap`ing the .blend file into virtual memory and directly
accessing it.
In my local testing, this speeds up loading the Dweebs file with all its
linked files from 19sec to 10sec (on Linux).
As far as I can see, this should be supported on Linux, OSX and BSD.
For Windows, a second code path uses `CreateFileMapping` and
`MapViewOfFile` to achieve the same result.
Reviewed By: mont29, brecht
Differential Revision: https://developer.blender.org/D8246
