This allows to generate source file that will
be injected in a predefined source dependance tree.
This allow much cleaner shader workflow where
all sources are explicitly referenced from the
main source file.
Pull Request: https://projects.blender.org/blender/blender/pulls/140047
Briefly about this change:
- OpenColorIO C-API is removed.
- The information about color spaces in ImBuf module is removed.
It was stored in global ListBase in colormanagement.cc.
- Both OpenColorIO and fallback implementation supports GPU drawing.
- Fallback implementation supports white point, RGB curves, etc.
- Removed check for support of GPU drawing in IMB.
Historically it was implemented in a separate library with C-API, this
is because way back C++ code needed to stay in intern. This causes all
sort of overheads, and even calls that are strictly considered bad
level.
This change moves OpenColorIO integration into a module within imbuf,
next to movie, and next to IMB_colormanagement which is the main user
of it. This allows to avoid copy of color spaces, displays, views etc
in the ImBuf: they were used to help quickly querying information to
be shown on the interface. With this change it can be stored in the
same data structures as what is used by the OpenColorIO integration.
While it might not be fully avoiding duplication it is now less, and
there is no need in the user code to maintain the copies.
In a lot of cases this change also avoids allocations done per access
to the OpenColorIO. For example, it is not needed anymore to allocate
image descriptor in a heap.
The bigger user-visible change is that the fallback implementation now
supports GLSL drawing, with the whole list of supported features, such
as curve mapping and white point. This should help simplifying code
which relies on color space conversion on GPU: there is no need to
figure out fallback solution in such cases. The only case when drawing
will not work is when there is some actual bug, or driver issue, and
shader has failed to compile.
The change avoids having an opaque type for color space, and instead
uses forward declaration. It is a bit verbose on declaration, but helps
avoiding unsafe type-casts. There are ways to solve this in the future,
like having a header for forward declaration, or to flatten the name
space a bit.
There should be no user-level changes under normal operation.
When building without OpenColorIO or the configuration has a typo or
is missing a fuller set of color management tools is applies (such as the
white point correction).
Pull Request: https://projects.blender.org/blender/blender/pulls/138433
See #129009 for context.
The preprocessor parses metadata and writes a header file containing
an inline function that inits the `GPUSource` with the metadata.
These header files are then included inside `gpu_shader_dependency.cc`.
This still keep the usage of the `metadata` enums and classes to avoid
pulling the whole blender module inside the preprocessor executable.
This speeds-up startup time in Debug build:
`gpu_shader_dependency_init`
- Before : 37ms
- After : 4ms
I didn't measure release, but it is unlikely to be noticeable (in the
order of 4ms > 1ms).
Pull Request: https://projects.blender.org/blender/blender/pulls/138070
This unify the C++ and GLSL codebase style.
The GLSL types are still in the backend compatibility
layers to support python shaders. However, the C++
shader compilation layer doesn't have them to enforce
correct type usage.
Note that this is going to break pretty much all PRs
in flight that targets shader code.
Rel #137261
Pull Request: https://projects.blender.org/blender/blender/pulls/137369
Potential fix for legacy AMD driver issue.
- Updating drivers using a clean install has proven to fix the issue as well.
As driver can leave parts of an older driver active what it actually the cau
- Solution comments out the `#line` by replacing the first two characters.
Pull Request: https://projects.blender.org/blender/blender/pulls/136231
Blender already had its own copy of OpenSubDiv containing some local fixes
and code-style. This code still used gl-calls. This PR updates the calls
to use GPU module. This allows us to use OpenSubDiv to be usable on other
backends as well.
This PR was tested on OpenGL, Vulkan and Metal. Metal can be enabled,
but Vulkan requires some API changes to work with loose geometry.
# Considerations
**ShaderCreateInfo**
intern/opensubdiv now requires access to GPU module. This to create buffers
in the correct context and trigger correct dispatches. ShaderCreateInfo is used
to construct the shader for cross compilation to Metal/Vulkan. However opensubdiv
shader caching structures are still used.
**Vertex buffers vs storage buffers**
Implementation tries to keep as close to the original OSD implementation. If
they used storage buffers for data, we will use GPUStorageBuf. If it uses vertex
buffers, we will use gpu::VertBuf.
**Evaluator const**
The evaluator cannot be const anymore as the GPU module API only allows
updating SSBOs when constructing. API could be improved to support updating
SSBOs.
Current implementation has a change to use reads out of bounds when constructing
SSBOs. An API change is in the planning to remove this issue. This will be fixed in
an upcoming PR. We wanted to land this PR as the visibility of the issue is not
common and multiple other changes rely on this PR to land.
Pull Request: https://projects.blender.org/blender/blender/pulls/135296
This PR migrates the subdiv_patch_evaluation_comp.glsl to use
shader create info.
The part of OSD that is used is included as a typedef source (osd_patch_basis.glsl).
Pull Request: https://projects.blender.org/blender/blender/pulls/134917
Previously, there was a `StringRef.copy` method which would copy the string into
the given buffer. However, it was not defined for the case when the buffer was
too small. It moved the responsibility of making sure the buffer is large enough
to the caller.
Unfortunately, in practice that easily hides bugs in builds without asserts
which don't come up in testing much. Now, the method is replaced with
`StringRef.copy_utf8_truncated` which has much more well defined semantics and
also makes sure that the string remains valid utf-8.
This also renames `unsafe_copy` to `copy_unsafe` to make the naming more similar
to `copy_utf8_truncated`.
Pull Request: https://projects.blender.org/blender/blender/pulls/133677
This allows to use `assert()` directly inside shader source.
The current implementation is just replacing it with a printf
that gives some information about the location of the assert.
It is not that much more helpful than a printf with a condition.
What is useful is that they are disabled by default. So they
can be sprinkle around and only turned on during debugging.
Adding shader name inside the assert report is not trivial and
could be added later (the shader name is only known at compile
time which is too late for string parsing).
Adding which drawcall produced the assert isn't trivial either.
It would require flushing the printf buffer for each drawcall
which might force synchronization and remove bugs.
Pull Request: https://projects.blender.org/blender/blender/pulls/129728
Avoid measuring the length of strings repeatedly by passing their
length along with their data with `StringRefNull`. Null termination
seems to be necessary still for passing the shader sources to OpenGL.
Though I doubt this is a bottleneck, it's still nice to avoid overhead from
string operations and this helps move in that direction.
Pull Request: https://projects.blender.org/blender/blender/pulls/127702
The goal is to reduce the startup time cost of
all of these parsing and string replacement.
All comments are now stripped at compile time.
This comment check added noticeable slowdown at
startup in debug builds and during preprocessing.
Put all metadatas between start and end token.
Use very simple parsing using `StringRef` and
hash all identifiers.
Move all the complexity to the preprocessor that
massagess the metadata into a well expected input
to the runtime parser.
All identifiers are compile time hashed so that no string
comparison is made at runtime.
Speed up the source loading:
- from 10ms to 1.6ms (6.25x speedup) in release
- from 194ms to 6ms (32.3x speedup) in debug
Follow up #129009
Pull Request: https://projects.blender.org/blender/blender/pulls/128927
Move most of the string preprocessing used for MSL
compatibility to `glsl_preprocess`.
Enforce some changes like matrix constructor and
array constructor to the GLSL codebase. This is
for C++ compatibility.
Additionally reduce the amount of code duplication
inside the compatibility code.
Pull Request: https://projects.blender.org/blender/blender/pulls/128634
This changes the include directive to use the standard C preprocessor
`#include` directive.
The regex to applied to all glsl sources is:
`pragma BLENDER_REQUIRE\((\w+\.glsl)\)`
`include "$1"`
This allow C++ linter to parse the code and allow easier codebase
traversal.
However there is a small catch. While it does work like a standard
include directive when the code is treated as C++, it doesn't when
compiled by our shader backends. In this case, we still use our
dependency concatenation approach instead of file injection.
This means that included files will always be prepended when compiled
to GLSL and a file cannot be appended more than once.
This is why all GLSL lib file should have the `#pragma once` directive
and always be included at the start of the file.
These requirements are actually already enforced by our code-style
in practice.
On the implementation, the source needed to be mutated to comment
the `#pragma once` and `#include`. This is needed to avoid GLSL
compiler error out as this is an extension that not all vendor
supports.
Rel #127983
Pull Request: https://projects.blender.org/blender/blender/pulls/128076
The code has nullptr checks to make sure it doesn't try to use freed or unitiallized variables.
However the when freeing the data, the code doesn't assign the pointer to be null.
Now set them to be null
Pull Request: https://projects.blender.org/blender/blender/pulls/125090
This allows much easier debugging of shader programs.
Usage is as simple as adding `printf` calls inside shaders.
example: `printf("Formating %d\n", my_var);`
Contrary to the `drw_print`, this is not limited
to draw manager shader dispatch/draws. It is compatible
with any shader inside blender.
Most notably, this doesn't need a viewport to display.
So this can be used to debug render pipeline.
Data formating is currently limited to only `%x`, `%d`,
`%u` and `%f`. This could be easily extended if this is
really needed.
There is no type checking, so values are directly reinterpreted
as specified by the printf format.
The current approach for making this work is to bind a
storage buffer inside `GPU_shader_bind`, making it
available to any shader that needs it. The storage buffer
is downloaded back to CPU after a frame or a render
step and the content printed to the console.
This scheduling means that you cannot rely on these printfs
to detect crashes. We could add a mode to force flushing
at shader binding to avoid this limitation.
The values are written from the shaders in binary form and
only formated on the CPU. This avoid issues with manual
printing like with `drw_print`.
Pull Request: https://projects.blender.org/blender/blender/pulls/125071
This implement the design of #118961.
- Add aliases in GLSL since theses types are
not supported.
- Add detection mechanism that prevents usage
inside shader shared code.
Check is only done in debug build to avoid slowing down
application startup.
Pull Request: https://projects.blender.org/blender/blender/pulls/119226
NDEBUG is part of the C standard and disables asserts. Only this will
now be used to decide if asserts are enabled.
DEBUG was a Blender specific define, that has now been removed.
_DEBUG is a Visual Studio define for builds in Debug configuration.
Blender defines this for all platforms. This is still used in a few
places in the draw code, and in external libraries Bullet and Mantaflow.
Pull Request: https://projects.blender.org/blender/blender/pulls/115774
The hash tables and vector blenlib headers were pulling many more
headers than they actually need, including the C base math header,
our C string API header, and the StringRef header. All of this
potentially slows down compilation and polutes autocomplete
with unrelated information.
Also remove the `ListBase` constructor for `Vector`. It wasn't used
much, and making it easy to use `ListBase` isn't worth it for the
same reasons mentioned above.
It turns out a lot of files depended on indirect includes of
`BLI_string.h` and `BLI_listbase.h`, so those are fixed here.
Pull Request: https://projects.blender.org/blender/blender/pulls/111801
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
This patch implements the Convert Color Space node for the realtime
compositor. A custom OCIO GpuShaderCreator was implemented to use the
ShaderCreateInfo in constructing the OCIO GPU processor shader. That
shader is then cached inside the cache manager and is invalidated when
the OCIO configuration changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/107878
The goal is to solve confusion of the "All rights reserved" for licensing
code under an open-source license.
The phrase "All rights reserved" comes from a historical convention that
required this phrase for the copyright protection to apply. This convention
is no longer relevant.
However, even though the phrase has no meaning in establishing the copyright
it has not lost meaning in terms of licensing.
This change makes it so code under the Blender Foundation copyright does
not use "all rights reserved". This is also how the GPL license itself
states how to apply it to the source code:
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software ...
This change does not change copyright notice in cases when the copyright
is dual (BF and an author), or just an author of the code. It also does
mot change copyright which is inherited from NaN Holding BV as it needs
some further investigation about what is the proper way to handle it.
Required by Metal backend for efficient shader compilation. EEVEE material
resource binding permutations now controlled via CreateInfo and selected
based on material options. Other existing CreateInfo's also modified to
ensure explicitness for depth-writing mode. Other missing bindings also
addressed to ensure full compliance with the Metal backend.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D16243
This patch moves the GLSL shaders and their infos to the compositor
module as decided by the EEVEE & Viewport module. This is a non
functional change.
Differential Revision: https://developer.blender.org/D16360
Reviewed By: Clement Foucault
This is a complete rewrite of the draw debug drawing module in C++.
It uses `GPUStorageBuf` to store the data to be drawn and use indirect
drawing. This makes it easier to do a mirror API for GPU shaders.
The C++ API class is exposed through `draw_debug.hh` and should be used
when possible in new code.
However, the debug drawing will not work for platform not yet supporting
`GPUStorageBuf`. Also keep in mind that this module must only be used
in debug build for performance and compatibility reasons.
This patch implements the necessary changes to the GPU module that are
needed by the realtime compositor.
A new function GPU_material_from_callbacks was added to construct a GPU
material from a number of callbacks. A callback to construct the
material graph by adding and linking the necessary GPU material nodes.
And the existing code generator callback. This essentially allows the
construction of GPU materials independent of node trees and without the
need to do any node tree localization.
A new composite source output to the code generator was added. This
output contains the serialization of nodes that are tagged with
GPU_NODE_TAG_COMPOSITOR, which are the nodes linked to the newly added
composite output links.
Two new GPU uniform setters were added for int2 and matrix3 types.
Shader create info now supports generated compute sources.
Shaders starting with gpu_shader_compositor are now considered part of
the shader library.
Additionally, two fixes were implemented. First, GPU setter node
de-duplication now appropriately increments the reference count of the
references resources. Second, unlinked sockets now get their value from
their associated GPU node stack instead of the socket itself.
Differential Revision: https://developer.blender.org/D14690
Reviewed By: Clement
