The main goal of these changes are to improve static (i.e. build-time)
checks on whether a given data can be allocated and freed with `malloc`
and `free` (C-style), or requires proper C++-style construction and
destruction (`new` and `delete`).
* Add new `MEM_malloc_arrayN_aligned` API.
* Make `MEM_freeN` a template function in C++, which does static assert on
type triviality.
* Add `MEM_SAFE_DELETE`, similar to `MEM_SAFE_FREE` but calling
`MEM_delete`.
The changes to `MEM_freeN` was painful and useful, as it allowed to fix a bunch
of invalid calls in existing codebase already.
It also highlighted a fair amount of places where it is called to free incomplete
type pointers, which is likely a sign of badly designed code (there should
rather be an API to destroy and free these data then, if the data type is not fully
publicly exposed). For now, these are 'worked around' by explicitly casting the
freed pointers to `void *` in these cases - which also makes them easy to search for.
Some of these will be addressed separately (see blender/blender!134765).
Finally, MSVC seems to consider structs defining new/delete operators (e.g. by
using the `MEM_CXX_CLASS_ALLOC_FUNCS` macro) as non-trivial. This does not
seem to follow the definition of type triviality, so for now static type checking in
`MEM_freeN` has been disabled for Windows. We'll likely have to do the same
with type-safe `MEM_[cm]allocN` API being worked on in blender/blender!134771
Based on ideas from Brecht in blender/blender!134452
Pull Request: https://projects.blender.org/blender/blender/pulls/134463
194e233d86 caused a discussion in the chat about the initialization
behavior of `MEM_new()`, and agreement was to not rely on
zero-initialization ever. Noted this in the API comment now.
Some people found the existing comment useful but it still left some
questions. Tried to clarify that now.
This is a crucial memory management function, it's important to have
behavior documented well, even if a full explanation is out-of-scope.
Also added another link in case people want to check more details.
Pull Request: https://projects.blender.org/blender/blender/pulls/134577
Add a new API to store data that is guaranteed to not be freed
before the memleak detector has run.
This will be used in next commit by the readfile code to improve
reporting on leaks from blendfile readingi process.
This is done by a two-layer approach:
A new templated `MEM_construct_leak_detection_data` allows to
create any type of data. Its ownership and lifetime are handled
internally, and guaranteed to not be destroyed before the memleak
detector has run.
Add a new template-based 'allocation string storage' system to
`intern/memutil`. This uses the new `Guardedalloc Persistent Storage`
system to store all 'complex' allocation messages, that cannot be
defined as literals.
Internally, the storage is done through an owning reference (a
`shared_ptr`) of the created data into a mutex-protected static
vector.
`MEM_init_memleak_detection` code ensures that this static storage
is created before the memleak detection data, so that it is destructed
after the memleak detector has ran.
The main container (`AllocStringStorageContainer`) is wrapping a
map of `{string -> AllocStringStorage<key_type, hash_type>}`.
The key is a storage identifier.
Each storage is also a map wrapped into a simple templated API
class (`AllocStringStorage`), where the values are the alloc strings,
and the keys type is defined by the user code.
Pull Request: https://projects.blender.org/blender/blender/pulls/125320
This commit will error (and abort if enabled) when trying to call
`MEM_freeN` (and related `MEM_dupallocN`, `MEM_reallocN` and
`MEM_recallocN` functions) with a pointer created the C++ way (i.e.
through `MEM_new`, or the guardedalloc-overloaded `new` operator).
To do so, it adds internal use only implementations for `malloc_alligned`
and `free`, which take an extra parameter indicating whether they are
dealing with data created/deleted the 'C++ way' (using `new`/`delete`
and similar).
The cpp-created data are flagged with the new
`MEMHEAD_FLAG_FROM_CPP_NEW`, either in the lower two-bytes len value for
lockfree allocator, or as a new flag member of the guarded allocator
header data.
The public `MEM_new`/`MEM_delete` template functions, and the
guardedalloc-overloaded versions of `new`/`delete` operators are updated
accordingly.
These changes have been successfully tested both with and without
`WITH_CXX_GUARDEDALLOC`.
NOTE: A lot of mismatches have already been fixed in `main` before merging
this change. There are likely some less easy to trigger ones still in our
codebase though.
Pull Request: https://projects.blender.org/blender/blender/pulls/123740
Almost certainly not an issue in current codebase (this 'copy' version
of `MEM_cnew` does not seem much used in the first place), but better be
consistent with the 'allocating' version.
Pull Request: https://projects.blender.org/blender/blender/pulls/123445
Ensure that the MemHead and MemHeadAligned are such that memory
allocation followed with the head offset keeps the allocation
aligned to at least MEM_MIN_CPP_ALIGNMENT.
Pull Request: https://projects.blender.org/blender/blender/pulls/120582
This `operator new` added in ecc3e78d78
are only called if the alignment is greater than `__STDCPP_DEFAULT_NEW_ALIGNMENT__`.
This is generally 8 or 16 depending on the platform. `MEM_mallocN` does
guarantee 16 byte alignment currently (in fact it's usually not 16 byte aligned
because of `MemHead`). Now `MEM_mallocN_aligned` is used with the default
alignment, even if we don't know that the type does not require it.
An alternative would be to pass the alignment to `MEM_CXX_CLASS_ALLOC_FUNCS`,
but that would be more intrusive.
Pull Request: https://projects.blender.org/blender/blender/pulls/118568
Previously, the alignment of structs that use `MEM_CXX_CLASS_ALLOC_FUNCS`
were not taken into account when doing the allocation. This can cause some data
to be mis-aligned and leads to crashes when cpu instructions or code expect the
data to be aligned.
The fix is to provide an overload of `operator new` that accepts the alignment as parameter.
More info: https://en.cppreference.com/w/cpp/language/new (search for `align_val_t`).
Pull Request: https://projects.blender.org/blender/blender/pulls/118526
For example
```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```
becomes
```
OIIOOutputDriver::~OIIOOutputDriver() {}
```
Saves quite some vertical space, which is especially handy for
constructors.
Pull Request: https://projects.blender.org/blender/blender/pulls/105594
`MEM_delete()` is designed for type safe destruction and freeing, void
pointers make that impossible.
Was reviewing a patch that was trying to free a C-style custom data
pointer this way. Apparently MSVC compiles this just fine, other
compilers error out. Make sure this is a build error on all platforms
with a useful message.
Seems to be introduced by 99e5024e97.
The crash is caused by the difference in the expected alignment
of the `uiPopupMenu` which is 16 bytes and the actual alignment
returned by the `MEM_mallocN()` which is 8 bytes due to the memory
head.
Now made it so that `MEM_new()` can be used for types with any
alignment.
Differential Revision: https://developer.blender.org/D16375
This is a more C++ friendly version MEM_calloc_arrayN, like MEM_cnew is for
MEM_callocN. For cases where data structures are still C and Vector or Array
don't work.
We have plenty of sorta generic functions, that allocate memory with
some generic name for debugging. When such a function is called and the
memory leaks, it may be unclear which call to it allocated the unfreed
memory (and thus which execution path leads to the leak).
The added function is only available if `NDEBUG` is not defined.
Differential Revision: https://developer.blender.org/D15605
Reviewed by: Sergey Sharybin, Bastien Montagne
Solved by introducing introducing a variant of MEM_cnew which behaves
as a copy-constructor for a trivial types.
Alternative approach would be to surround DNA structs with clang/gcc
diagnostics push/modify/pop so that implicitly defined constructors
and copy operators are allowed to access deprecated fields.
The downside of the DNA approach is that it will require some way to
easily apply diagnostics modifications to many structs, which is not
possible currently.
The newly added MEM_cnew has other good usecases, so is easiest to
use this route, at least for now.
Differential Revision: https://developer.blender.org/D14356
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Initialization with `MEM_new()` depends a lot on the initialization rules
of C++, which are not obvious. Calling it with no arguments to be passed
to the constructor may cause the resulting object to be implicitly 0
initialized (or parts of it). This can have an impact on performance
sensitive code, so it's something to document.
Alternatively we could enforce default initialization (as opposed to the
value initalization that happens now), but this could cause
uninitialized memory in a way that isn't obvious either. This is a
possible source of bugs, so Jacques and I decided against it.
Using the `MEM_*` API from C++ code was a bit annoying:
* When converting C to C++ code, one often has to add a type cast on
returned `void *`. That leads to having the same type name three times
in the same line. This patch reduces the amount to two and removes the
`sizeof(...)` from the line.
* The existing alternative of using `OBJECT_GUARDED_NEW` looks a out
of place compared to other allocation methods. Sometimes
`MEM_CXX_CLASS_ALLOC_FUNCS` can be used when structs are defined
in C++ code. It doesn't look great but it's definitely better. The downside
is that it makes the name of the allocation less useful. That's because
the same name is used for all allocations of a type, independend of
where it is allocated.
This patch introduces three new functions: `MEM_new`, `MEM_cnew` and
`MEM_delete`. These cover the majority of use cases (array allocation is
not covered).
The `OBJECT_GUARDED_*` macros are removed because they are not
needed anymore.
Differential Revision: https://developer.blender.org/D13502
- Nest compositor pages under the compositor module
- Nest GUI, DNA/RNA & externformats modules under Blender.
- Remove modules from intern which no longer exist.
- Add intern modules (atomic, eigen, glew-mx, libc_compat, locale,
numaapi, rigidbody, sky, utfconv).
- Use 'intern_' prefix for intern modules since some of the modules
use generic terms such as locale & atomic.
Add comment explaining `MEM_dupallocN` is NULL-safe, in that it returns
NULL when it receives a NULL pointer. This is currently true for both
implementations of the function (`MEM_lockfree_dupallocN` and
`MEM_guarded_dupallocN`), and will be expected of other implementations
as well.
No functional changes.
Previously the only way to use lockfree implementation was to start
executable and never switch to guarded allocator.
Surely, it is not possible to switch implementation once any allocation
did happen, but some tests are desired to test lock-free implementation
of the allocator. Those tests did not operate properly because the main
entry point of tests are forcing guarded allocator to help catching
bugs.
This change makes it possible for those tests to ensure they do operate
on lock-free implementation.
There is no functional changes here, preparing boilerplate for an
upcoming work on the allocator tests themselves.
This adds a new `--debug-exit-on-error` flag. When it is set, Blender
will abort with a non-zero exit code when there are internal errors.
Currently, "internal errors" includes memory leaks detected by
guardedalloc and error/fatal log entries in clog.
The new flag is passed to Blender in various places where automated
tests are run. Furthermore, the `--debug-memory` flag is used in tests,
because that makes the verbose output more useful, when dealing
with memory leaks.
Reviewers: brecht, sergey
Differential Revision: https://developer.blender.org/D8665
The abbreviation 'init' is brief, unambiguous and already used
in thousands of places, also initialize is often accidentally
written with British spelling.
When definining static variables that own memory, you should
use the "construct on first use" idiom. Otherwise, you'll get
a warning when Blender exits.
More details are provided in D8354.
Differential Revision: https://developer.blender.org/D8354
