Refactor the OCIO config to use the updated OpenColorIO 2.x recommended
format, which places spaces like sRGB and Display P3 into a separate section
dedicated to display-referred colorspaces, and puts view transforms in their
own section as well.
View transform are no longer declared as color spaces, they are now rerouted
to cie_xyz_d65_interchange, and then from there go to the display colorspace.
Also remove the dedicated reverse LUT for AgX. Since view transforms are
no longer colorspaces, using them in a `- !<ColorSpaceTransform>` line no
longer works. The original way of re-using the inverse definition with
`- !<ColorSpaceTransform>` no longer works, it gets a bit troublesome. AgX
has never been able to perfectly invert, not even with this inverse LUT, so
might as well remove it.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/145519
* Add ACES SDR to HDR displays
* Add ACES reference gamut compression look.
* Name non-HDR AgX for HDR displays "AgX - SDR", consistent with ACES and
makes it more clear that this may not be the one you want for HDR. This
required updating test blend files.
* Mark all non-sRGB view transform colorspaces as inactive, so they don't
pollute the colorspaces list. The HDR ones were already inactive.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/145820
* Bundled ICC profiles for display spaces supported by Blender, and
embed them in the image file when saving.
* Verified to work for PNG, TIFF, JPEG and WebP, but not all file
formats support this.
* No ICC profile is written for sRGB currently. It would be a matter
of adding an icc file, however this may be a breaking change for
some use cases.
* Fix save as render of EXR files not properly changing the image
colorspace to match.
Uses CC0 licensed ICC files from the Compact ICC Profiles project.
This does not include support for saving HDR images. While there exist
ICC profiles for PQ, they are not well supported and the preferred method
for HDR is to write CICP tags. However OpenImageIO support for this is
still under development.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/144565
These are built into OpenColorIO, and will start working well with the upcoming
wide gamut display changes.
The ACES 2.0 view transform provides an alternative to AgX and Filmic, with
similar purpose. It is also useful for those working in an ACES pipeline when
the full ACES OpenColorIO config is not needed.
Rec.2100-PQ display is a HDR display, and will be what you select when wanting
to see HDR (and wide gamut) colors, and export them to HDR video.
Ref #144911
Pull Request: https://projects.blender.org/blender/blender/pulls/144565
HDR video files are properly read into Blender, and can be rendered out
of Blender.
HDR video reading / decoding:
- Two flavors of HDR are recognized, based on color related video
metadata: "PQ" (Rec.2100 Perceptual Quantizer, aka SMPTE 2084) and
"HLG" (Rec.2100 Hybrid-Log-Gamma, aka ARIB STD B67). Both are read
effectively into floating point images, and their color space
transformations are done through OpenColorIO.
- The OCIO config shipped in Blender has been extended to contain
Rec.2100-PQ and Rec.2100-HLG color spaces.
- Note that if you already had a HDR video in sequencer or movie clip,
it would have looked "incorrect" previously, and it will continue to
look incorrect, since it already has "wrong" color space assigned to
it. Either re-add it (which should assign the correct color space),
or manually change the color space to PQ or HLG one as needed.
HDR video writing / encoding"
- For H.265 and AV1 the video encoding options now display the HDR mode.
Similar to reading, there are PQ and HLG HDR mode options.
- Reference white is assumed to be 100 nits.
- YUV uses "full" ("PC/jpeg") color range.
- No mastering display metadata is written into the video file, since
generally that information is not known inside Blender.
More details and screenshots in the PR.
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/120033
This PR does four things, and shouldn't have visible impact to users.
* Use fewer decimal places for D65 white point, to match e.g. sRGB and
Rec.2020 standards.
* Fix mistake in grayscale matrix for Grayscale look and False Color view
transform.
* Remove unused luminance_compensation_srgb/p3
* Improve AgX Log's matrix comment
Pull Request: https://projects.blender.org/blender/blender/pulls/141027
- "Log Encoding with Chroma inset and rotation": add "of primaries" in
the description of the AgX Log color space to better explain the
operation, based on wording in !106355.
- Remove a few double spaces.
- Make Line Art title case everywhere, to convey it's the system /
brand / product name and not the generic concept.
- "Copy Absolute coordinates or Normal vector" -> "of Normal Vector":
typo.
- "Makes a link between selected output in input sockets" ->
"Make...", "output and input": typo.
- "Purge Unused Data From This File" -> "from this": title case as per
HIGs.
- GPencil -> Grease Pencil: no reason to use an abbreviation here.
- "Around Current Frame" -> "Around Frame": actual name of the
onion-skinning method.
- "... (layer height for layer tool, i.e.)" -> "(i.e. the layer height
for the layer tool)": put "i.e." at the start of the sentence.
- Expand description of toe-in stereo camera option.
- "Children collections their parent-collection-specific settings" ->
"Children collections with their...": typo.
- "Generate vertex weights base on..." -> "based on" : typo, lower
case.
- Expand description of GP modifier properties, based on their mesh
counterparts.
- "AEnvelope" -> "Envelope": typo.
- "Falloff type the feather" -> "of the feather": typo.
- "usually make transition as long as effect strip": rephrase.
- "When disabled a users extensions directory is created" -> "a
user's": typo.
- "successfull" -> "successful": typo.
- "Remove all attributes... a single wildcard (*).": remove trailing
".".
- "..., use "Save Preferences."": remove trailing ".".
Some issues reported by Marina Veselkova and Tamar Mebonia.
Pull Request: https://projects.blender.org/blender/blender/pulls/120649
A tone mapper designed specifically for PBR color accuracy, to get sRGB
colors in the output render that match as faithfully as possible the input
sRGB baseColor under gray-scale lighting. This is aimed toward product
photography use cases, where the scene is well-exposed and HDR color values
are mostly restricted to small specular highlights.
Fixes#118824: Proposal: add a view transform for Khronos PBR Neutral Tone Mapper
Co-authored-by: Emmett Lalish <elalish@google.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/118936
Since changing the reference color space to XYZ E, there was some imprecision
in color space conversions.
Change the XYZ D65 to Linear Rec.709 conversion matrix very slightly to avoid
values rounding the wrong way.
Updated matrix provided by Zijun Zhou.
Pull Request: https://projects.blender.org/blender/blender/pulls/115767
Seems that the ExponentWithLinearTransform with offset=0 does not render
properly on GPU on Apply Silicon GPUs. Likely, it is possible to use plain
ExponentTransform in this case, which should also be faster.
Candidate for 4.0.1 corrective release.
Pull Request: https://projects.blender.org/blender/blender/pulls/114853
AgX exhibited some banding-like artifacts that were due to being
approximated with a 3D LUT. This commit resolves that by increasing
the LUT resolution enough to mitigate the artifacts and make them
unnoticeable.
Additionally:
- The previous LUTs were written in a space-inefficient way, using
e.g. "0.000000" instead of "0". The new LUTs are written more
efficiently, avoiding quite as dramatic a file size increase as
usually accompanies 3D LUT resolution increases.
- The previous LUTs included output values greater than 1.0, which was
both incorrect for a tone mapper, and also pointless since Blender
immediately clips them anyway. The new LUTs clip to 1.0. This also
allows the more efficient writing to squeeze even more space savings
out of the LUTs.
- The previous inverse AgX LUT contained NaNs. Those have been
replaced with 0.0 in the new inverse LUT.
Note that due to discrepancies between the LUTs previously provided
to Blender and the AgX scripts that were later published, the color
transform in these LUTs are slightly different. But they are close, and
equivalently good.
Pull Request: https://projects.blender.org/blender/blender/pulls/113253
This PR shrinks the Filmic LUT's size down without causing noticeable
difference, making the overall LUT package size smaller. There should be no
functional change on the user end.
Pull Request: https://projects.blender.org/blender/blender/pulls/111380
On a user level this view transform provides much better handling of colors in
the over-exposed areas.
With this configuration the following display devices are available, including
AgX view transform for them:
* sRGB
* Display P3
* Rec.1886
* Rec.2020
NOTE: There is no Filmic view transform available for the newly added display
devices.
AgX also brings an implementation of False Colors view transform, which replaces
Filmic-based, and is available for all display devices.
The backward compatibility is preserved. The new files will default to AgX view
transform, which makes it non-forward compatible.
More technical details is available in the original PR #106355.
Please note that the PR has been split into more incremental changes when
was landing.
Pull Request: https://projects.blender.org/blender/blender/pulls/111099
This is the next step towards AgX view transform project.
This is a sanitization pass over the existing color space names, as well as
addition of some new spaces. For example, the name Linear makes no sense when
there are a bunch of Linear spaces. An aliases are included for backwards
compatibility.
Some of the space descriptions and family tags are also updated., which also
involves in putting `False Color` to `inactive_colorspaces` instead of using
`family: display` filtering.
The spaces are now:
- Linear components of the display spaces
- Linear Rec.709
- Linear DCI-P3 D65
- Linear Rec.2020
- Linear ACES spaces
- ACES2065-1
- ACEScg (Changed from `Linear ACEScg` to `ACEScg` since ACEScg already
implied a linear transfer function, otherwise using `Linear AP1 ACES`
might make more sense. Same goes for ACES2065-1)
- Linear FilmLight E-Gamut
This is for AgX's LUT input encoding. It can potentially be useful for interop.
- Display spaces
- sRGB
- Display P3
- Rec.1886
- Rec.2020
- Filmic Components
- Filmic Log
- Filmic sRGB
- False Color
Ref #110685
Pull Request: https://projects.blender.org/blender/blender/pulls/110941
The change is motivated by using the actual CIE standard, preparing for the
Spectral Cycles, and making followup AgX integration more straight forward.
On a user level this change renames XYZ space to Linear CIE-XYZ D65, and
introduces Linear CIE-XYZ E.
The aliases are configured so that the backward compatibility is preserved.
The forward compatibility will be achieved by committing an alias name to the
3.6 release branch.
Ref #110685
Use OCIO's built-in functionality for sRGB which allows to have un-clipped
range of the input colors. The lookup tables are now removed.
Should be no functional changes on the user side.
However, it is possible that this change does not preserve exact per-pixel
value due to difference in precision.
Ref #110685
Pull Request: https://projects.blender.org/blender/blender/pulls/110712
This is a Step 1 of the AgX project integration, which is focused on cleaning
up legacy aspects of the configuration.
Delete all not longer used colorspaces and their LUT files. This includes:
nuke_rec709, lg10, XYZ display device and its standard view colorspace.
The nuke_rec709, lg10 were not used, and the XYZ display device was initially
added to support output for a DCP compliant J2K files. Unfortunately, some of
the crucial transform was missing from this initial configuration, so often a
custom OCIO configuration was needed. With the addition of Filmic and now with
AgX this really became not-so-usable use-case. Also, there is no display device
which is XYZ and on which Blender can realistically run.
The None display device has duplicated functionality as the Raw view in sRGB
display. Its need originated for the compatibility with legacy "No Color
Management" option in Blender. It is also missing in the fallback color
management implementation. So, now use sRGB with Raw view to replicate the
old behavior. There will be a separate do-version commit after this one to help
transitioning the current files to the new configuration.
The Raw and Non-Color are duplicates of each other, there is no need to have
both of them. Since users are more familiar with Non-Color, and it's also the
one assigned as the data role, will keep Non-Color and remove Raw. For the
compatibility the Raw is an alias of the Non-Color space.
Ref #110685
A typical use case is when you want to render with the Filmic view transform, but
composite an existing image in the background that should not be affected by the
view transform.
With this colorspace it's possible to do an inverse Filmic transform, render
everything in scene linear space, and then apply the Filmic transform again.
This is pretty basic in that this is not going to take into account the full view
transform including looks, curves and exposure. But it can be helpful anyway.
Ref T68926
This affected loading of EXR files with set to Linear ACES colorspace, as
well as the sky texture for in some custom OpenColorIO configurations.
Use the builtin OpenColorIO transform from ACES AP0 to XYZ D65 to fix this.
The Blender/Cycles XYZ color space has a D65 white point instead of E, and
this was not correctly accounted for both in the OpenColor config reading code
and the bundled config.
This meant that since the OpenColorIO v2 upgrade, the Linear ACES color space
was not working correctly, and other OpenColorIO configs defining
aces_interchange were not interpreted correctly.
Ref T84819
Build System
============
This is an API breaking new version, and the updated code only builds with
OpenColorIO 2.0 and later. Adding backwards compatibility was too complicated.
* Tinyxml was replaced with Expat, adding a new dependency.
* Yaml-cpp is now built as a dependency on Unix, as was already done on Windows.
* Removed currently unused LCMS code.
* Pystring remains built as part of OCIO itself, since it has no good build system.
* Linux and macOS check for the OpenColorIO verison, and disable it if too old.
Ref D10270
Processors and Transforms
=========================
CPU processors now need to be created to do CPU processing. These are cached
internally, but the cache lookup is not fast enough to execute per pixel or
texture sample, so for performance these are now also exposed in the C API.
The C API for transforms will no longer be needed afer all changes, so remove
it to simplify the API and fallback implementation.
Ref D10271
Display Transforms
==================
Needs a bit more manual work constructing the transform. LegacyViewingPipeline
could also have been used, but isn't really any simpler and since it's legacy
we better not rely on it.
We moved more logic into the opencolorio module, to simplify the API. There is
no need to wrap a dozen functions just to be able to do this in C rather than C++.
It's also tightly coupled to the GPU shader logic, and so should be in the same
module.
Ref D10271
GPU Display Shader
==================
To avoid baking exposure and gamma into the GLSL shader and requiring slow
recompiles when tweaking, we manually apply them in the shader. This leads
to some logic duplicaton between the CPU and GPU display processor, but it
seems unavoidable.
Caching was also changed. Previously this was done both on the imbuf and
opencolorio module levels. Now it's all done in the opencolorio module by
simply matching color space names. We no longer use cacheIDs from OpenColorIO
since computing them is expensive, and they are unlikely to match now that
more is baked into the shader code.
Shaders can now use multiple 2D textures, 3D textures and uniforms, rather
than a single 3D texture. So allocating and binding those adds some code.
Color space conversions for blending with overlays is now hardcoded in the
shader. This was using harcoded numbers anyway, if this every becomes a
general OpenColorIO transform it can be changed, but for now there is no
point to add code complexity.
Ref D10273
CIE XYZ
=======
We need standard CIE XYZ values for rendering effects like blackbody emission.
The relation to the scene linear role is based on OpenColorIO configuration.
In OpenColorIO 2.0 configs roles can no longer have the same name as color
spaces, which means our XYZ role and colorspace in the configuration give an
error.
Instead use the new standard aces_interchange role, which relates scene linear
to a known scene referred color space. Compatibility with the old XYZ role is
preserved, if the configuration file has no conflicting names.
Also includes a non-functional change to the configuraton file to use an
XYZ-to-ACES matrix instead of REC709-to-ACES, makes debugging a little easier
since the matrix is the same one we have in the code now and that is also
found easily in the ACES specs.
Ref D10274
Change to false color are based on artist feedback:
* Even 2.5 EV differences down to -10 EV.
* Even 2.167 EV differences up to +6.5 EV.
* Slightly below middle grey biased toward cyan.
* Slightly above middle grey biased toward yellow.
* Converts to nonlinear luma for more accuracy.
* Replace Log view transform with Filmic Log.
* Remove Rec.709, DCI-P3 displays that were incomplete and outdated.
* Remove outdated RRT and Film transforms, replaced by Filmic.
* Remove camera responsive curves that don't work with HDR colors.
* Rename Default view transform to Standard.
We're breaking compatibility now for 2.80, so that we can add future
improvements on a clean config.
Part of the code was contributed by George Vogiatzis in D4782.
Differential Revision: https://developer.blender.org/D4900
* Use simple default view transform for color pickers, as Filmic does not work
well for all types of colors. We better handle this with an option and tagging
of colors as emissive or albedo like.
* For solid/workbench we also no longer use Filmic, as there is not enough contrast
and it's not really needed since this is not physically based lighting.
* For lookdev always take into account the view transform and look. Other view
settings like exposure are only taken into account if scene lighting is used,
since these are often dependent on scene light intensity.
Fixes T61022, T57649, T59363.
In 2d655d3 the color picker was changed to use display space HSV values.
This works ok for a simple sRGB EOTF, but fails with view transforms like
Filmic where display space V 1.0 maps to RGB 16.292.
Instead we now use the color_picking role from the OCIO config when
converting from RGB to HSV in the color picker. This role is set to sRGB
in the default OCIO config.
This color space fits the following requirements:
* It is approximately perceptually linear, so that the HSV numbers and
the HSV cube/circle have an intuitive distribution.
* It has the same gamut as the scene linear color space.
* Color picking values 0..1 map to scene linear values in the 0..1 range,
so that picked albedo values are energy conserving.
This ignores the scene color managment view settings for solid mode and
lookdev when not using scene lights and world. The scene settings are
intended for tweaking renders and should not affect studio lighting and
matcaps.
There may be cases where a simple sRGB transform is better than Filmic
and we could add configuration for this. Not sure if it really matters
and it may be better if we just assume matcaps and studiolights are all
created for one view transform.
Differential Revision: https://developer.blender.org/D3569
I've limited it to just the RGB<->XYZ stuff for now, correct image handling is the next step.
Reviewers: brecht, sergey
Differential Revision: https://developer.blender.org/D3478
