RNA raw types were missing for the int8_t, uchar (uint8_t),
ushort (uint16_t), int64_t and uint64_t DNA types types.
This adds the missing RNA raw types for all DNA types that can have
raw access.
Functional Changes
Properties with one of the new unsigned raw types will raise a Python
OverflowError in foreach_getset when attempting to read a negative
integer from bpy_prop_collection.foreach_set(). This is similar to the
existing behaviour of providing a Python int which is too large to
represent as a C long. The existing foreach_getset code will print
the OverflowError and then raise a TypeError instead.
CPython's signed integer parsing functions accept numeric types that are
not the Python int instances by calling their __index__() method.
CPython's unsigned integer parsing functions, however, only accept
Python int instances. To make foreach_getset accept the same
numeric types for unsigned raw types as it already accepts for signed
raw types, the unsigned integer parsing functions in py_capi_utils.h
have been updated to also call the __index__() method when given an
argument which is not a Python int instance.
Because the new unsigned integer parsing in foreach_getset is using
the PyC_ family of functions, which perform their own overflow checks,
the existing signed integer parsing has also been updated to use the
PyC_ family of functions for signed integer parsing. Previously,
OverflowError would only have been raised when the parsed integer was
too large to fit in a C long. With this patch, OverflowError will be
raised whenever the parsed integer is too large to fit in the property's
raw type. Integer properties already have well-defined maximum and
minimum values which should fit within the property's raw type, and enum
properties have a fixed number of values they can take depending on
their items. The bigger change here, is that setting bool properties
which use PROP_RAW_BOOLEAN will now only accept 0/False and
1/True.
Now that PROP_RAW_CHAR parsing is using PyC_Long_AsU8,
signed char buffers ("b" format) have been updated to no longer be
considered compatible with PROP_RAW_CHAR, matching the behaviour of
the other unsigned types only being considered compatible with unsigned
buffer formats.
The int64_t and uint64_t types can currently only be used by bool
properties (see IS_DNATYPE_BOOLEAN_COMPAT and the other macros in
RNA_define.hh), but bool properties only have raw access when they do
not use a bitmask and it doesn't make much sense to use an entire 64
bits just for a single bool property, so PROP_RAW_INT64 and
PROP_RAW_UINT64 are expected to be unused.
Performance Changes
Providing raw types allows for faster access through
rna_access.cc#rna_raw_access, especially when a buffer compatible with
the property's raw type is passed through from
bpy_rna.cc#foreach_getset.
Before this patch, the bpy.types.Keyframe.handle_left_type property
did not have raw access, so foreach_getset would fall back to
PROP_RAW_INT being the compatible type and then use the slower loop in
rna_raw_access.
With this patch, the bpy.types.Keyframe.handle_left_type property has
raw access with the PROP_RAW_UINT8 type. Using a buffer compatible
with this raw type can use the faster memcpy loop in
rna_raw_access. Using a Python list will iterate the list into an
array whose type matches PROP_RAW_UINT8, which will also use the
faster memcpy loop in #rna_raw_access.
Pull Request: https://projects.blender.org/blender/blender/pulls/115761
RNA raw types were missing for the int8_t, uchar (uint8_t),
ushort (uint16_t), int64_t and uint64_t DNA types types.
This adds the missing RNA raw types for all DNA types that can have
raw access.
Functional Changes
Properties with one of the new unsigned raw types will raise a Python
OverflowError in foreach_getset when attempting to read a negative
integer from bpy_prop_collection.foreach_set(). This is similar to the
existing behaviour of providing a Python int which is too large to
represent as a C long. The existing foreach_getset code will print
the OverflowError and then raise a TypeError instead.
CPython's signed integer parsing functions accept numeric types that are
not the Python int instances by calling their __index__() method.
CPython's unsigned integer parsing functions, however, only accept
Python int instances. To make foreach_getset accept the same
numeric types for unsigned raw types as it already accepts for signed
raw types, the unsigned integer parsing functions in py_capi_utils.h
have been updated to also call the __index__() method when given an
argument which is not a Python int instance.
Because the new unsigned integer parsing in foreach_getset is using
the PyC_ family of functions, which perform their own overflow checks,
the existing signed integer parsing has also been updated to use the
PyC_ family of functions for signed integer parsing. Previously,
OverflowError would only have been raised when the parsed integer was
too large to fit in a C long. With this patch, OverflowError will be
raised whenever the parsed integer is too large to fit in the property's
raw type. Integer properties already have well-defined maximum and
minimum values which should fit within the property's raw type, and enum
properties have a fixed number of values they can take depending on
their items. The bigger change here, is that setting bool properties
which use PROP_RAW_BOOLEAN will now only accept 0/False and
1/True.
Now that PROP_RAW_CHAR parsing is using PyC_Long_AsU8,
signed char buffers ("b" format) have been updated to no longer be
considered compatible with PROP_RAW_CHAR, matching the behaviour of
the other unsigned types only being considered compatible with unsigned
buffer formats.
The int64_t and uint64_t types can currently only be used by bool
properties (see IS_DNATYPE_BOOLEAN_COMPAT and the other macros in
RNA_define.hh), but bool properties only have raw access when they do
not use a bitmask and it doesn't make much sense to use an entire 64
bits just for a single bool property, so PROP_RAW_INT64 and
PROP_RAW_UINT64 are expected to be unused.
Performance Changes
Providing raw types allows for faster access through
rna_access.cc#rna_raw_access, especially when a buffer compatible with
the property's raw type is passed through from
bpy_rna.cc#foreach_getset.
Before this patch, the bpy.types.Keyframe.handle_left_type property
did not have raw access, so foreach_getset would fall back to
PROP_RAW_INT being the compatible type and then use the slower loop in
rna_raw_access.
With this patch, the bpy.types.Keyframe.handle_left_type property has
raw access with the PROP_RAW_UINT8 type. Using a buffer compatible
with this raw type can use the faster memcpy loop in
rna_raw_access. Using a Python list will iterate the list into an
array whose type matches PROP_RAW_UINT8, which will also use the
faster memcpy loop in #rna_raw_access.
Pull Request: https://projects.blender.org/blender/blender/pulls/115761
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
`PyArg_ParseTupleAndKeywords` does not initialize variable corresponding
to optional arguments that aren't passed by Python. Thanks to Germano for
initial investigation.
Add support for enum values in ID properties.
This is needed for the "Menu Switch" node implementation (#113445) which
relies on ID properties for the top-level modifier UI.
Enums items can optionally be added to the UI data of integer
properties. Each property stores a full set of the enum items to keep
things simple.
Enum items can be added to properties using the `id_properties_ui`
function in the python API. A detailed example can be found in the
`bl_pyapi_idprop.py` test.
There is currently no support yet for editing enum items through the UI.
This is because the "Edit Property" feature is implemented entirely
through a single operator (`WM_OT_properties_edit`) and its properties.
Buttons to add/remove/move items would be operators changing another
operator's properties. A refactor of the custom properties UI is likely
required to make this work.
Pull Request: https://projects.blender.org/blender/blender/pulls/114362
Code using #PyObject_GetBuffer was combining the `PyBUF_FORMAT` and
`PyBUF_SIMPLE` flags, but the documentation specifies that
`PyBUF_FORMAT` can be |'d to any of the flags except `PyBUF_SIMPLE`
because the latter already implies format `B` (unsigned bytes).
The flags in such cases have been replaced with
`PyBUF_ND | PyBUF_FORMAT`, which has the additional requirement that the
buffer must provide it's `shape` field.
This fixes `memoryview` objects raising a `BufferError` when requested,
due to the invalid combination of flags making them be considered
invalid buffers when they would otherwise be valid.
Ref: !106697
The last good commit was 8474716abb.
After this commits from main were pushed to blender-v4.0-release. These are
being reverted.
Commits a4880576dc from to b26f176d1a that happend afterwards were meant for
4.0, and their contents is preserved.
BGL is deprecated and will not work on Metal devices. Although the
inital plan was to remove it in Blender 4.0, We don't see any harm
to still have it in the code-base until OpenGL itself is deprecated.
Add-on developers are warned when using the BGL module that the
add-on/script will not work on all platforms.
There are still some limitations inside the GPU module that needs
a more friendly API. This API isn't clear at this time.
Pull Request: https://projects.blender.org/blender/blender/pulls/112579
Replacing PyErr_Print with PyErr_Display in [0] caused string errors
not to display because PyErr_Display doesn't normalize the exception.
Normalizing before displaying the error resolves this.
[0]: 6a0f98aeef
Some of the comments for exiting were outdated & vague.
Add additional comments to clarify out SystemExit, sys.exit() & atexit
are used to handle exit requests from Python within Blender.
Non-matching calls to PyErr_Fetch/Restore cause a leak in v3.12,
so ensure calls are symmetrical or avoid where possible.
Simplify extraction of the exception buffer.
- Only overwrite the stderr (the stdio isn't used).
- Simplify pyc_exception_buffer_handle_system_exit usage.
- Remove goto's.
Also simplify calling conventions for PyC_ExceptionBuffer functions.
- They must be called when an error has occurred.
- Always return a string, never null since a null return value would
only happened in rare/unexpected cases which wasn't being checked
for by some callers, leading to potential crashes.
The internals of PyErr_Print/PyErr_Display & the traceback module
are different enough that it's not likely the Python traceback
implementation will be used.
While in practice sys.modules has __main__ module, the purpose of
PyC_MainModule_Backup/Restore is to temporarily override __main__.
Running code outside of the text editor (callbacks for timers for e.g.)
could remove the __main__ module which would then crash when running
a script from the text editor.
- Account for new member in _PyArg_Parser.
- Many Python op-codes have been removed.
For the moment these are disabled in is_opcode_secure.
Some should be added back as intrinsics, noted in code-comments.
New code exposing ID Pointers CustomProperties to the UI (b3c7f3c8a9)
would cause invalid data access when calling `as_dict()` on py-defined ID
pointers ID properties, since these have no ID type defined by default.
The code of `as_dict()` will now forcefully generate a valid ID type
value, either by using the type of the currently assigned ID, or falling
back to the Object ID type.
Found while opening some lighting production files from the Pets
Project.
Customprops to IDs are supported since years through code, but were
never exposed directly in the UI of customporperties.
This commit mainly:
* Adds a new `DATA_BLOCK` type to UI customprops types.
* Exposes the existing `id_type` settings to python API.
Pull Request: https://projects.blender.org/blender/blender/pulls/110458
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Use PyC_ParseUnicodeAsBytesAndSize parser instead of "s" / "z" type
specifier. This relates to #111033, resolving Python exceptions which
causes icons not to load (for e.g.).
Now bytes are also supported as path arguments.
Expose PyC_UnicodeAsBytesAndSize using a parser which PyArg_ParseTuple
can use. This should be used with PyArg_ParseTuple instead of "s" type
specifier when passing in file-system path arguments.
PyC_ParseUnicodeAsBytesAndSize_OrNone has been included to match "z".
Using ClangBuildAnalyzer on the whole Blender build, it was pointing
out that BLI_math.h is the heaviest "header hub" (i.e. non tiny file
that is included a lot).
However, there's very little (actually zero) source files in Blender
that need "all the math" (base, colors, vectors, matrices,
quaternions, intersection, interpolation, statistics, solvers and
time). A common use case is source files needing just vectors, or
just vectors & matrices, or just colors etc. Actually, 181 files
were including the whole math thing without needing it at all.
This change removes BLI_math.h completely, and instead in all the
places that need it, includes BLI_math_vector.h or BLI_math_color.h
and so on.
Change from that:
- BLI_math_color.h was included 1399 times -> now 408 (took 114.0sec
to parse -> now 36.3sec)
- BLI_simd.h 1403 -> 418 (109.7sec -> 34.9sec).
Full rebuild of Blender (Apple M1, Xcode, RelWithDebInfo) is not
affected much (342sec -> 334sec). Most of benefit would be when
someone's changing BLI_simd.h or BLI_math_color.h or similar files,
that now there's 3x fewer files result in a recompile.
Pull Request #110944
The cleanup of blenkernel last weeks , caused the house of cards to
collapse on top of bf_gpu's shader_builder, which is off by default
but used on a daily basis by the rendering team.
Given the fixes forward in #110394 ran into a ODR violation in OSL that
was hiding there for years, I don't see another way forward without
impeding the rendering teams productivity for "quite a while" as there
is no guarantee the OSL issue would be the end of it.
the only way forward appears to be back.
this reverts :
19422044eda670b53abe0f541db97cbe516e8c813e88a2f44c4e64b772f59547e7a31707fe6c5a57
The problematic commit was 07fe6c5a57
as blenkernel links most of blender, it's a bit of a link order issue
magnet. Given all these commits stack, it's near impossible to revert
just that one without spending a significant amount of time resolving
merge conflicts. 99% of that work was automated, so easier to just
revert all of them, and re-do the work, than it is to deal with the
merge conflicts.
Pull Request: https://projects.blender.org/blender/blender/pulls/110438
