BLI code for enums that are meant to be used as "bit flags" defined
an ENUM_OPERATORS macro in BLI_utildefines.h. This cleans up things
related to said macro:
- Move it out into a separate BLI_enum_flags.hh header, instead of
"random bag of things" that is the current place,
- Update it to no longer need manual indication of highest individual
bit value. This originally was added in a31a87f89 (2020 Oct), in
order to silence some UBSan warnings that were coming
from GPU related structures (looking at current GPU code, I don't
think this is happening anymore). However, that caused actual
user-visible bugs due to incorrectly specified max. enum bit value,
and today 14% of all usages have incorrect highest individual
bit value spelled out.
- I have reviewed all usages of operator ~ and none of them are
used for directly producing a DNA-serialized value; all the
usages are for masking out other bits for which the new ~
behavior that just flips all bits is fine.
- Make the macro define flag_is_set() function to ease check of bits
that are set in C++ enum class cases; update existing cases to use
that instead of three other ways that were used.
Pull Request: https://projects.blender.org/blender/blender/pulls/148230
Improve mmap handling of IO errors on WIN32.
Make MMAP gracefully handle IO errors on Windows by replacing the
mapping with zeros using a vectored exception handler when an
EXCEPTION_IN_PAGE_ERROR is raised. This is similar to how such errors
are handled on non-Windows platforms.
On Windows, this is implemented by first creating a placeholder
allocation and then mapping the file into it. When an error occurs, the
exception handler unmaps the file, keeping the placeholder intact, and
creates an anonymous mapping into it, after which execution can
continue.
Since some required functions don't exist on older Windows versions,
the error handling will only work on Windows 10, version 1803 or newer.
Ref !139739
Since VSE HDR preview was implemented (18110744a2), the preview image
is put into RGBA 16F format "color texture", and the overlay
(checkerboard, grids etc.) is put into RGBA 8U format "overlay texture".
This was rendering the color image as NON premultiplied alpha in
the viewport color texture, which is not what happens in other places
(e.g. image space).
Later commit (c279d894db) fixed transparency of the overlay texture,
in order to make pure emissive (color non zero, alpha zero) colors
from EXR files show up in VSE preview. However that exposed the issue
that underlying color texture is not premultiplied.
Fix this by premultiplying the output color in gpu_shader_display_transform
that is used by VSE preview. This is done at the end of the shader
isntead of using fixed function blending, since premultiplication needs
to happen in a way that preserves pure-emissive colors.
Comparison images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/147892
The function name `for_each_callback` is ambigous (what are we iterating?)
and also doesn't align with the naming of other "for each" functions in Blender
which usually are named `foreach_<thing>` like `IndexMask::foreach_index`
or `animrig::foreach_fcurve_in_action` etc.
Pull Request: https://projects.blender.org/blender/blender/pulls/147440
Based on extensive testing, this gives matching HDR brighness across most
application, with both PNG and AVIF. Video remains at 100 nits as that
appears to tbe convention there.
This is implemented by adding two modified PQ and HLG color spaces to the
OCIO config on startup, and for the specific cases of image save and loaded
these will replace the regular PQ and HLG color spaces.
This was chosen rather than adding them as color spaces in the OCIO config,
so that it can work for any config with appropriate interop IDs, including
the ACES config. Additionally, it would be unclear how to make this work with
view + display transforms, we wouldn't want to burden the users with having
to pick a different display depending if they are saving images or video.
Ref #145855, #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146888
In Render properties > Color Management > Display.
* Off: Directly output image as produced by OpenColorIO. This is not correct
in general, but may be used when the system configuration and actual display
device is known to match the chosen display.
* Automatic: Display images consistent with most other applications, to preview
images and video for export. A best effort is made to emulate the chosen
display on the actual display device.
The option is grayed out when the current OpenColorIO config and display/view
does not support emulation.
Ref #145022, #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146808
Previously it was only working for the single layer case. For multipart
we write the colorspace in each part. For single part we write the first
non-data colorspace, and hope data passes will be identified based on channel
name like Blender does (e.g. XYZ instead of RGB).
Reading is unchanged and still the same as before, in that it only reads the
colorspace from the first part. There is only one color space per image
datablock, so we can not store anything more currently. In practice it
would be unusual for all passes in a file not to either have the same
colorspace or be data.
All the compositor file output test images were updated to include the
metadata, so that the test will check if the metadata is there.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146809
New "Interleave" option in image format settings, enabled by default in old
files and disabled by default in new files to write multi-part files.
By not storing all channels interleaved, it is possible for other applications
to load individual passes more efficiently. This might also work better with
compression.
We follow the OpenEXR docs in that the channel name is the full
Layer.Pass.Channel rather than just Channel. Some other renderers
(e.g. Houdini Karma) write just the channel and that is what our reading
code assumed so far.
I've verified that Nuke can read the multipart EXR files produced by Blender,
including multiview and cryptomatte.
Fix#123727
Pull Request: https://projects.blender.org/blender/blender/pulls/146650
* Read directly into ExrChannel, eliminate intermediate MultiViewChannelName
* Pass multiple channels to EXR writer together in IMB_exr_add_channels
* Avoid various channel name parsing by passing components separately
* Simplify logic for writing multichannel and multiview metadata
* Remove unused global EXR handle storage
* No longer use void pointer for EXR handle.
* Use blender::Vector and std::string.
* Slightly reshuffle code so multipart support will have smaller diff.
* Add various comments.
Pull Request: https://projects.blender.org/blender/blender/pulls/146650
The reading code currently assumes that the part name may be for example
"ViewLayer.Combined" and then the channel within the part may be "R". For
example Houdini writes files like this.
However according to the docs the part name should be ignored and the
channel name in each part should be complete like "ViewLayer.Combined.R".
https://openexr.com/en/latest/MultiViewOpenEXR.html
To support both cases we now only prepend the part name if it's not already
there.
Pull Request: https://projects.blender.org/blender/blender/pulls/146650
### Color Management: Support image CICP read/write
The CICP support will enable reading and writing HDR PNGs, following
the recently released PNG spec:
https://www.w3.org/TR/png-3/#cICP-chunk
More useful for exporting to the web with reasonable file size would
be AVIF support, but that will have to wait for the next library update
in Blender 5.1 to add libheif.
Note this will only starting working once the 5.0 libraries have landed,
which should happen somewhere in the next week.
Pull Request: https://projects.blender.org/blender/blender/pulls/145612
And make it the default in the compositing Convert Color Space node. This is
useful to create compositing nodes that give the same result regardless of
the working space, particularly for node group assets. For example you might
convert from the working space to a log space, do some color correction
operations and convert back.
I'm less certain about including this also for images. I can imagine use
cases for that too, for example with an Image node in the compositor that is
known to read back the EXR sequence written from the current scene. But it's
a bit more obscure and there is some potential for using this when it's not
appropriate. Still there is nothing that automatically selects it, so it seems
fine.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146598
No longer use it for display of images without View as Render, color swatches
and color picker, and 3D viewport in solid mode. Instead go directly to extended
sRGB matching the system graphics buffer.
We still use it for setting the image colorspace after Save as Render, and for
color value inspection in the image editor and tooltips.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146487
Rather than relying on an untonemapped view, use the OpenColorIO v2
display color spaces. This provides a generic way of converting between
display referred color spaces using the cie_xyz_d65_interchange role
as the intermediate display linear space.
On older configs without such display color spaces, display emulation
will be disabled.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/146487
This is the first step of moving the create infos
back inside shader sources.
All info files are now treated as source files.
However, they are not considered in the include tree
yet. This will come in another following PR.
Each shader source file now generate a `.info` file
containing only the create info declarations.
This renames all info files so that they do not
conflict with their previous versions that were
copied (non-generated).
Pull Request: https://projects.blender.org/blender/blender/pulls/146676
Before this change, any accumulated errors from OpenImageIO would be
lost and a nonsensical use of `errno` would be used instead.
An example of the new error log:
```
00:23.188 image.write | ERROR OpenImageIO write failed: sgi image resolution may not exceed 65535x65535, you asked for 65536x1
```
Pull Request: https://projects.blender.org/blender/blender/pulls/146554
There was an incorrect assumption that the display colorspace of a
view + display transform uniquely identifies it. But with modern OCIO
configs there is no longer a unique display colorspace per view.
The same wrong assumption was made in the code that refreshes the image
editor after save, and that was fixed as well.
Ref #145519
Pull Request: https://projects.blender.org/blender/blender/pulls/146449
* Fix: OpenColorIO config defined working space disappears from menu. It
was incorrectly using the current scene linear instead of default space.
* Warn users about missing working space.
* Don't detect lin_ap0_scene working space, we don't offer it in the menu.
Pull Request: https://projects.blender.org/blender/blender/pulls/146149
Some of the color space settings are not validated on load to be set to
a valid values. To deal with those cases better bake it so color management
module does not make assumption on validness of the input data.
This is an oversight in the !140729.
Pull Request: https://projects.blender.org/blender/blender/pulls/146084
For most players this appears to made no difference. But one case it fixes
is that with this metadata, on Youtube with SDR, 10bit PQ looks the same as
12bit PQ, where it was previously too desaturated.
10bit and 12bit HLG was already consistent without this metadata and there
appears to be no impact, though it remains inconsistent with PQ.
It seems good practice to be explicit about the mastering display regardless.
The way we get the max luminance is not ideal though. We get it from the
view transform name containing "HDR XXXX nits" as used in the Blender and
ACES configs. There is currently no way to query this information from
OpenColorIO.
In the future we can add a user control for this. For now if we can not
determine the max luminance this way, we don't write mastering display
metadata. The video player will then make some default assumption for tone
mapping, often 1000 nits.
Pull Request: https://projects.blender.org/blender/blender/pulls/144378
When using the default Blender config use what was saved in the startup blend.
But when using a custom config it's likely the scene linear space from that config
is wanted. And not using it can lead to some confusing situations, where you end
up with an ACES config and the Linear Rec.709 working space.
Pull Request: https://projects.blender.org/blender/blender/pulls/145884
* Store scene linear to XYZ conversion matrix in each blend file, along
with the colorspace name. The matrix is the source of truth. The name
is currently only used for error logging about unknown color spaces.
* Add Working Space option in color management panel, to change the
working space for the entire blend file. Changing this will pop up
a dialog, with a default enabled option to convert all colors in
the blend file to the new working space. Note this is necessarily only
an approximation.
* Link and append automatically converts to the color space of the main
open blend file.
* There is builtin support for Rec.709, Rec.2020 and ACEScg working spaces,
in addition to the working space of custom OpenColorIO configs.
* Undo of working space for linked datablocks isn't quite correct when going
to a smaller gamut working space. This can be fixed by reloading the file
so the linked datablocks are reloaded.
Compatibility with blend files saved with a custom OpenColorIO config
is tricky, as we can not detect this.
* We assume that if the blend file has no information about the scene
linear color space, it is the default one from the active OCIO config.
And the same for any blend files linked or appended. This is effectively
the same behavior as before.
* Now that there is a warning when color spaces are missing, it is more
likely that a user will notice something is wrong and only save the
blend file with the correct config active.
* As no automatic working space conversion happens on file load, there is
an opportunity to correct things by changing the working space with
"Convert Colors" disabled. This can also be scripted for all blend files
in a project.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/145476
There is a bad const cast here, but seems to work ok in practice. Creating a
whole new config would be tricky, though something we should support in the
future for other reasons.
Pull Request: https://projects.blender.org/blender/blender/pulls/145476
Both were largely or completely single threaded.
They are used in various places, but testing their usage in VSE
compositor modifier branch (!139634), applying a default "do nothing"
compositor modifier on a 1080p image (on Ryzen 5950X):
51.4ms -> 12.2ms
Details about IMB_byte_from_float:
- No longer allocate a full new float buffer, instead do all work in
a local small (32KB size, half of typical L1 cache) job-local buffer.
- Previous code was doing un-premultiply + OCIO + premultiply
+ un-premultiply again. That is pointless; just do
un-premultiply once.
Details about IMB_float_from_byte / IMB_float_from_byte_ex:
- Remove incorrect code around"allocate float buffer outside of image
buffer" since it was not actually true to begin with.
- Inside threaded part, do color space conversion and premultiply at
once per-scanline, so that data stays in CPU caches more.
Pull Request: https://projects.blender.org/blender/blender/pulls/145716
The tooltips for fonts in the file browser, while in list view, now
shows either no preview or a blank white square (depending if you
viewed them with thumbnails in the current session). This PR corrects
that to always show a preview. It also shows this preview while in
thumbnail view.
Pull Request: https://projects.blender.org/blender/blender/pulls/145685
