griefith/test
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

RNA: Add missing raw types for DNA types

Browse Source 
RNA raw types were missing for the int8_t, uchar (uint8_t),
ushort (uint16_t), int64_t and uint64_t DNA types types.

This adds the missing RNA raw types for all DNA types that can have
raw access.

Functional Changes

Properties with one of the new unsigned raw types will raise a Python
OverflowError in foreach_getset when attempting to read a negative
integer from bpy_prop_collection.foreach_set(). This is similar to the
existing behaviour of providing a Python int which is too large to
represent as a C long. The existing foreach_getset code will print
the OverflowError and then raise a TypeError instead.

CPython's signed integer parsing functions accept numeric types that are
not the Python int instances by calling their __index__() method.
CPython's unsigned integer parsing functions, however, only accept
Python int instances. To make foreach_getset accept the same
numeric types for unsigned raw types as it already accepts for signed
raw types, the unsigned integer parsing functions in py_capi_utils.h
have been updated to also call the __index__() method when given an
argument which is not a Python int instance.

Because the new unsigned integer parsing in foreach_getset is using
the PyC_ family of functions, which perform their own overflow checks,
the existing signed integer parsing has also been updated to use the
PyC_ family of functions for signed integer parsing. Previously,
OverflowError would only have been raised when the parsed integer was
too large to fit in a C long. With this patch, OverflowError will be
raised whenever the parsed integer is too large to fit in the property's
raw type. Integer properties already have well-defined maximum and
minimum values which should fit within the property's raw type, and enum
properties have a fixed number of values they can take depending on
their items. The bigger change here, is that setting bool properties
which use PROP_RAW_BOOLEAN will now only accept 0/False and
1/True.

Now that PROP_RAW_CHAR parsing is using PyC_Long_AsU8,
signed char buffers ("b" format) have been updated to no longer be
considered compatible with PROP_RAW_CHAR, matching the behaviour of
the other unsigned types only being considered compatible with unsigned
buffer formats.

The int64_t and uint64_t types can currently only be used by bool
properties (see IS_DNATYPE_BOOLEAN_COMPAT and the other macros in
RNA_define.hh), but bool properties only have raw access when they do
not use a bitmask and it doesn't make much sense to use an entire 64
bits just for a single bool property, so PROP_RAW_INT64 and
PROP_RAW_UINT64 are expected to be unused.

Performance Changes

Providing raw types allows for faster access through
rna_access.cc#rna_raw_access, especially when a buffer compatible with
the property's raw type is passed through from
bpy_rna.cc#foreach_getset.

Before this patch, the bpy.types.Keyframe.handle_left_type property
did not have raw access, so foreach_getset would fall back to
PROP_RAW_INT being the compatible type and then use the slower loop in
rna_raw_access.

With this patch, the bpy.types.Keyframe.handle_left_type property has
raw access with the PROP_RAW_UINT8 type. Using a buffer compatible
with this raw type can use the faster memcpy loop in
rna_raw_access. Using a Python list will iterate the list into an
array whose type matches PROP_RAW_UINT8, which will also use the
faster memcpy loop in #rna_raw_access.

Pull Request: https://projects.blender.org/blender/blender/pulls/115761
RNA raw types were missing for the int8_t, uchar (uint8_t),
ushort (uint16_t), int64_t and uint64_t DNA types types.

This adds the missing RNA raw types for all DNA types that can have
raw access.

Functional Changes

Properties with one of the new unsigned raw types will raise a Python
OverflowError in foreach_getset when attempting to read a negative
integer from bpy_prop_collection.foreach_set(). This is similar to the
existing behaviour of providing a Python int which is too large to
represent as a C long. The existing foreach_getset code will print
the OverflowError and then raise a TypeError instead.

CPython's signed integer parsing functions accept numeric types that are
not the Python int instances by calling their __index__() method.
CPython's unsigned integer parsing functions, however, only accept
Python int instances. To make foreach_getset accept the same
numeric types for unsigned raw types as it already accepts for signed
raw types, the unsigned integer parsing functions in py_capi_utils.h
have been updated to also call the __index__() method when given an
argument which is not a Python int instance.

Because the new unsigned integer parsing in foreach_getset is using
the PyC_ family of functions, which perform their own overflow checks,
the existing signed integer parsing has also been updated to use the
PyC_ family of functions for signed integer parsing. Previously,
OverflowError would only have been raised when the parsed integer was
too large to fit in a C long. With this patch, OverflowError will be
raised whenever the parsed integer is too large to fit in the property's
raw type. Integer properties already have well-defined maximum and
minimum values which should fit within the property's raw type, and enum
properties have a fixed number of values they can take depending on
their items. The bigger change here, is that setting bool properties
which use PROP_RAW_BOOLEAN will now only accept 0/False and
1/True.

Now that PROP_RAW_CHAR parsing is using PyC_Long_AsU8,
signed char buffers ("b" format) have been updated to no longer be
considered compatible with PROP_RAW_CHAR, matching the behaviour of
the other unsigned types only being considered compatible with unsigned
buffer formats.

The int64_t and uint64_t types can currently only be used by bool
properties (see IS_DNATYPE_BOOLEAN_COMPAT and the other macros in
RNA_define.hh), but bool properties only have raw access when they do
not use a bitmask and it doesn't make much sense to use an entire 64
bits just for a single bool property, so PROP_RAW_INT64 and
PROP_RAW_UINT64 are expected to be unused.

Performance Changes

Providing raw types allows for faster access through
rna_access.cc#rna_raw_access, especially when a buffer compatible with
the property's raw type is passed through from
bpy_rna.cc#foreach_getset.

Before this patch, the bpy.types.Keyframe.handle_left_type property
did not have raw access, so foreach_getset would fall back to
PROP_RAW_INT being the compatible type and then use the slower loop in
rna_raw_access.

With this patch, the bpy.types.Keyframe.handle_left_type property has
raw access with the PROP_RAW_UINT8 type. Using a buffer compatible
with this raw type can use the faster memcpy loop in
rna_raw_access. Using a Python list will iterate the list into an
array whose type matches PROP_RAW_UINT8, which will also use the
faster memcpy loop in #rna_raw_access.

Pull Request: https://projects.blender.org/blender/blender/pulls/115761
This commit is contained in:
Thomas Barlow
2024-01-10 18:19:24 +01:00
committed by Brecht Van Lommel
parent dda8d0920a
commit 992ec6487b
6 changed files with 173 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
source/blender/makesrna/RNA_types.hh
View File

@@ -446,6 +446,11 @@ enum RawPropertyType {
PROP_RAW_BOOLEAN,
PROP_RAW_DOUBLE,
PROP_RAW_FLOAT,
PROP_RAW_UINT8,
PROP_RAW_UINT16,
PROP_RAW_INT64,
PROP_RAW_UINT64,
PROP_RAW_INT8,
};
struct RawArray {

20
source/blender/makesrna/intern/makesrna.cc
View File

@@ -1968,10 +1968,22 @@ static void rna_set_raw_property(PropertyDefRNA *dp, PropertyRNA *prop)
prop->rawtype = PROP_RAW_CHAR;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "int8_t")) {
prop->rawtype = PROP_RAW_INT8;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "uchar")) {
prop->rawtype = PROP_RAW_UINT8;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "short")) {
prop->rawtype = PROP_RAW_SHORT;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "ushort")) {
prop->rawtype = PROP_RAW_UINT16;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "int")) {
prop->rawtype = PROP_RAW_INT;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
@@ -1984,6 +1996,14 @@ static void rna_set_raw_property(PropertyDefRNA *dp, PropertyRNA *prop)
prop->rawtype = PROP_RAW_DOUBLE;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "int64_t")) {
prop->rawtype = PROP_RAW_INT64;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
else if (STREQ(dp->dnatype, "uint64_t")) {
prop->rawtype = PROP_RAW_UINT64;
prop->flag_internal |= PROP_INTERN_RAW_ACCESS;
}
}
static void rna_set_raw_offset(FILE *f, StructRNA *srna, PropertyRNA *prop)

40
source/blender/makesrna/intern/rna_access.cc
View File

@@ -4586,9 +4586,18 @@ int RNA_property_collection_raw_array(
case PROP_RAW_CHAR: \
var = (dtype)((char *)raw.array)[a]; \
break; \
case PROP_RAW_INT8: \
var = (dtype)((int8_t *)raw.array)[a]; \
break; \
case PROP_RAW_UINT8: \
var = (dtype)((uint8_t *)raw.array)[a]; \
break; \
case PROP_RAW_SHORT: \
var = (dtype)((short *)raw.array)[a]; \
break; \
case PROP_RAW_UINT16: \
var = (dtype)((uint16_t *)raw.array)[a]; \
break; \
case PROP_RAW_INT: \
var = (dtype)((int *)raw.array)[a]; \
break; \
@@ -4601,6 +4610,12 @@ int RNA_property_collection_raw_array(
case PROP_RAW_DOUBLE: \
var = (dtype)((double *)raw.array)[a]; \
break; \
case PROP_RAW_INT64: \
var = (dtype)((int64_t *)raw.array)[a]; \
break; \
case PROP_RAW_UINT64: \
var = (dtype)((uint64_t *)raw.array)[a]; \
break; \
default: \
var = (dtype)0; \
} \
@@ -4613,9 +4628,18 @@ int RNA_property_collection_raw_array(
case PROP_RAW_CHAR: \
((char *)raw.array)[a] = char(var); \
break; \
case PROP_RAW_INT8: \
((int8_t *)raw.array)[a] = int8_t(var); \
break; \
case PROP_RAW_UINT8: \
((uint8_t *)raw.array)[a] = uint8_t(var); \
break; \
case PROP_RAW_SHORT: \
((short *)raw.array)[a] = short(var); \
break; \
case PROP_RAW_UINT16: \
((uint16_t *)raw.array)[a] = uint16_t(var); \
break; \
case PROP_RAW_INT: \
((int *)raw.array)[a] = int(var); \
break; \
@@ -4628,6 +4652,12 @@ int RNA_property_collection_raw_array(
case PROP_RAW_DOUBLE: \
((double *)raw.array)[a] = double(var); \
break; \
case PROP_RAW_INT64: \
((int64_t *)raw.array)[a] = int64_t(var); \
break; \
case PROP_RAW_UINT64: \
((uint64_t *)raw.array)[a] = uint64_t(var); \
break; \
default: \
break; \
} \
@@ -4639,8 +4669,14 @@ int RNA_raw_type_sizeof(RawPropertyType type)
switch (type) {
case PROP_RAW_CHAR:
return sizeof(char);
case PROP_RAW_INT8:
return sizeof(int8_t);
case PROP_RAW_UINT8:
return sizeof(uint8_t);
case PROP_RAW_SHORT:
return sizeof(short);
case PROP_RAW_UINT16:
return sizeof(uint16_t);
case PROP_RAW_INT:
return sizeof(int);
case PROP_RAW_BOOLEAN:
@@ -4649,6 +4685,10 @@ int RNA_raw_type_sizeof(RawPropertyType type)
return sizeof(float);
case PROP_RAW_DOUBLE:
return sizeof(double);
case PROP_RAW_INT64:
return sizeof(int64_t);
case PROP_RAW_UINT64:
return sizeof(uint64_t);
default:
return 0;
}

61
source/blender/python/generic/py_capi_utils.cc
View File

@@ -1636,6 +1636,32 @@ bool PyC_RunString_AsString(const char *imports[],
# pragma GCC diagnostic ignored "-Wtype-limits"
#endif
/* #PyLong_AsUnsignedLong, unlike #PyLong_AsLong, does not fall back to calling #PyNumber_Index
* when its argument is not a `PyLongObject` instance. To match parsing signed integer types with
* #PyLong_AsLong, this function performs the #PyNumber_Index fallback, if necessary, before
* calling #PyLong_AsUnsignedLong. */
static ulong pyc_Long_AsUnsignedLong(PyObject *value)
{
if (value == nullptr) {
/* Let PyLong_AsUnsignedLong handle error raising. */
return PyLong_AsUnsignedLong(value);
}
if (PyLong_Check(value)) {
return PyLong_AsUnsignedLong(value);
}
/* Call `__index__` like PyLong_AsLong. */
PyObject *value_converted = PyNumber_Index(value);
if (value_converted == nullptr) {
/* A `TypeError` will have been raised. */
return ulong(-1);
}
ulong to_return = PyLong_AsUnsignedLong(value_converted);
Py_DECREF(value_converted);
return to_return;
}
int PyC_Long_AsBool(PyObject *value)
{
const int test = _PyLong_AsInt(value);
@@ -1682,7 +1708,7 @@ int16_t PyC_Long_AsI16(PyObject *value)
uint8_t PyC_Long_AsU8(PyObject *value)
{
const ulong test = PyLong_AsUnsignedLong(value);
const ulong test = pyc_Long_AsUnsignedLong(value);
if (UNLIKELY(test == ulong(-1) && PyErr_Occurred())) {
return uint8_t(-1);
}
@@ -1695,7 +1721,7 @@ uint8_t PyC_Long_AsU8(PyObject *value)
uint16_t PyC_Long_AsU16(PyObject *value)
{
const ulong test = PyLong_AsUnsignedLong(value);
const ulong test = pyc_Long_AsUnsignedLong(value);
if (UNLIKELY(test == ulong(-1) && PyErr_Occurred())) {
return uint16_t(-1);
}
@@ -1708,7 +1734,7 @@ uint16_t PyC_Long_AsU16(PyObject *value)
uint32_t PyC_Long_AsU32(PyObject *value)
{
const ulong test = PyLong_AsUnsignedLong(value);
const ulong test = pyc_Long_AsUnsignedLong(value);
if (UNLIKELY(test == ulong(-1) && PyErr_Occurred())) {
return uint32_t(-1);
}
@@ -1719,9 +1745,32 @@ uint32_t PyC_Long_AsU32(PyObject *value)
return uint32_t(test);
}
/* Inlined in header:
* PyC_Long_AsU64
*/
/* #PyLong_AsUnsignedLongLong, unlike #PyLong_AsLongLong, does not fall back to calling
* #PyNumber_Index when its argument is not a `PyLongObject` instance. To match parsing signed
* integer types with #PyLong_AsLongLong, this function performs the #PyNumber_Index fallback, if
* necessary, before calling #PyLong_AsUnsignedLongLong. */
uint64_t PyC_Long_AsU64(PyObject *value)
{
if (value == nullptr) {
/* Let PyLong_AsUnsignedLongLong handle error raising. */
return uint64_t(PyLong_AsUnsignedLongLong(value));
}
if (PyLong_Check(value)) {
return uint64_t(PyLong_AsUnsignedLongLong(value));
}
/* Call `__index__` like PyLong_AsLongLong. */
PyObject *value_converted = PyNumber_Index(value);
if (value_converted == nullptr) {
/* A `TypeError` will have been raised. */
return uint64_t(-1);
}
uint64_t to_return = uint64_t(PyLong_AsUnsignedLongLong(value_converted));
Py_DECREF(value_converted);
return to_return;
}
#ifdef __GNUC__
# pragma warning(pop)

10
source/blender/python/generic/py_capi_utils.h
View File

@@ -302,12 +302,14 @@ int32_t PyC_Long_AsI32(PyObject *value);
int64_t PyC_Long_AsI64(PyObject *value);
#endif
/* Unlike Python's #PyLong_AsUnsignedLong and #PyLong_AsUnsignedLongLong, these unsigned integer
* parsing functions fall back to calling #PyNumber_Index when their argument is not a
* `PyLongObject`. This matches Python's signed integer parsing functions which also fall back to
* calling #PyNumber_Index. */
uint8_t PyC_Long_AsU8(PyObject *value);
uint16_t PyC_Long_AsU16(PyObject *value);
uint32_t PyC_Long_AsU32(PyObject *value);
#if 0 /* inline */
uint64_t PyC_Long_AsU64(PyObject *value);
#endif
/* inline so type signatures match as expected */
Py_LOCAL_INLINE(int32_t) PyC_Long_AsI32(PyObject *value)
@@ -318,10 +320,6 @@ Py_LOCAL_INLINE(int64_t) PyC_Long_AsI64(PyObject *value)
{
return (int64_t)PyLong_AsLongLong(value);
}
Py_LOCAL_INLINE(uint64_t) PyC_Long_AsU64(PyObject *value)
{
return (uint64_t)PyLong_AsUnsignedLongLong(value);
}
/* utils for format string in `struct` module style syntax */
char PyC_StructFmt_type_from_str(const char *typestr);

54
source/blender/python/intern/bpy_rna.cc
View File

@@ -5408,13 +5408,16 @@ static bool foreach_compat_buffer(RawPropertyType raw_type, int attr_signed, con
const char f = format ? *format : 'B'; /* B is assumed when not set */
switch (raw_type) {
case PROP_RAW_CHAR:
case PROP_RAW_INT8:
if (attr_signed) {
return (f == 'b') ? true : false;
}
else {
return (f == 'B') ? true : false;
}
case PROP_RAW_CHAR:
case PROP_RAW_UINT8:
return (f == 'B') ? true : false;
case PROP_RAW_SHORT:
if (attr_signed) {
return (f == 'h') ? true : false;
@@ -5422,6 +5425,8 @@ static bool foreach_compat_buffer(RawPropertyType raw_type, int attr_signed, con
else {
return (f == 'H') ? true : false;
}
case PROP_RAW_UINT16:
return (f == 'H') ? true : false;
case PROP_RAW_INT:
if (attr_signed) {
return (f == 'i') ? true : false;
@@ -5435,6 +5440,15 @@ static bool foreach_compat_buffer(RawPropertyType raw_type, int attr_signed, con
return (f == 'f') ? true : false;
case PROP_RAW_DOUBLE:
return (f == 'd') ? true : false;
case PROP_RAW_INT64:
if (attr_signed) {
return (f == 'q') ? true : false;
}
else {
return (f == 'Q') ? true : false;
}
case PROP_RAW_UINT64:
return (f == 'Q') ? true : false;
case PROP_RAW_UNSET:
return false;
}
@@ -5505,16 +5519,25 @@ static PyObject *foreach_getset(BPy_PropertyRNA *self, PyObject *args, int set)
item = PySequence_GetItem(seq, i);
switch (raw_type) {
case PROP_RAW_CHAR:
((char *)array)[i] = char(PyLong_AsLong(item));
((char *)array)[i] = char(PyC_Long_AsU8(item));
break;
case PROP_RAW_INT8:
((int8_t *)array)[i] = PyC_Long_AsI8(item);
break;
case PROP_RAW_UINT8:
((uint8_t *)array)[i] = PyC_Long_AsU8(item);
break;
case PROP_RAW_SHORT:
((short *)array)[i] = short(PyLong_AsLong(item));
((short *)array)[i] = short(PyC_Long_AsI16(item));
break;
case PROP_RAW_UINT16:
((uint16_t *)array)[i] = PyC_Long_AsU16(item);
break;
case PROP_RAW_INT:
((int *)array)[i] = int(PyLong_AsLong(item));
((int *)array)[i] = int(PyC_Long_AsI32(item));
break;
case PROP_RAW_BOOLEAN:
((bool *)array)[i] = int(PyLong_AsLong(item)) != 0;
((bool *)array)[i] = bool(PyC_Long_AsBool(item));
break;
case PROP_RAW_FLOAT:
((float *)array)[i] = float(PyFloat_AsDouble(item));
@@ -5522,6 +5545,12 @@ static PyObject *foreach_getset(BPy_PropertyRNA *self, PyObject *args, int set)
case PROP_RAW_DOUBLE:
((double *)array)[i] = double(PyFloat_AsDouble(item));
break;
case PROP_RAW_INT64:
((int64_t *)array)[i] = PyC_Long_AsI64(item);
break;
case PROP_RAW_UINT64:
((uint64_t *)array)[i] = PyC_Long_AsU64(item);
break;
case PROP_RAW_UNSET:
/* Should never happen. */
BLI_assert_msg(0, "Invalid array type - set");
@@ -5576,9 +5605,18 @@ static PyObject *foreach_getset(BPy_PropertyRNA *self, PyObject *args, int set)
case PROP_RAW_CHAR:
item = PyLong_FromLong(long(((char *)array)[i]));
break;
case PROP_RAW_INT8:
item = PyLong_FromLong(long(((int8_t *)array)[i]));
break;
case PROP_RAW_UINT8:
item = PyLong_FromLong(long(((uint8_t *)array)[i]));
break;
case PROP_RAW_SHORT:
item = PyLong_FromLong(long(((short *)array)[i]));
break;
case PROP_RAW_UINT16:
item = PyLong_FromLong(long(((uint16_t *)array)[i]));
break;
case PROP_RAW_INT:
item = PyLong_FromLong(long(((int *)array)[i]));
break;
@@ -5591,6 +5629,12 @@ static PyObject *foreach_getset(BPy_PropertyRNA *self, PyObject *args, int set)
case PROP_RAW_BOOLEAN:
item = PyBool_FromLong(long(((bool *)array)[i]));
break;
case PROP_RAW_INT64:
item = PyLong_FromLongLong(((int64_t *)array)[i]);
break;
case PROP_RAW_UINT64:
item = PyLong_FromUnsignedLongLong(((uint64_t *)array)[i]);
break;
default: /* PROP_RAW_UNSET */
/* Should never happen. */
BLI_assert_msg(0, "Invalid array type - get");