With the shift to GPU-driven rendering pipeline,
the SSBO vertex fetch paradigm used to
implement workbench shadows on Metal
instead of utilising the geometry shader
path no longer worked correctly.
This is because the draw submission
required vertex amplification up-front,
based on the expected output geometry
amount for a given input geometry.
This patch aims to resolve this
issue through addition of API to
enable the features within the
GPU driven pipeline.
Co-authored-by: Michael Parkin-White <mparkinwhite@apple.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/113498
Including <iostream> or similar headers is quite expensive, since it
also pulls in things like <locale> and so on. In many BLI headers,
iostreams are only used to implement some sort of "debug print",
or an operator<< for ostream.
Change some of the commonly used places to instead include <iosfwd>,
which is the standard way of forward-declaring iostreams related
classes, and move the actual debug-print / operator<< implementations
into .cc files.
This is not done for templated classes though (it would be possible
to provide explicit operator<< instantiations somewhere in the
source file, but that would lead to hard-to-figure-out linker error
whenever someone would add a different template type). There, where
possible, I changed from full <iostream> include to only the needed
<ostream> part.
For Span<T>, I just removed print_as_lines since it's not used by
anything. It could be moved into a .cc file using a similar approach
as above if needed.
Doing full blender build changes include counts this way:
- <iostream> 1986 -> 978
- <sstream> 2880 -> 925
It does not affect the total build time much though, mostly because
towards the end of it there's just several CPU cores finishing
compiling OpenVDB related source files.
Pull Request: https://projects.blender.org/blender/blender/pulls/111046
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
The goal is to solve confusion of the "All rights reserved" for licensing
code under an open-source license.
The phrase "All rights reserved" comes from a historical convention that
required this phrase for the copyright protection to apply. This convention
is no longer relevant.
However, even though the phrase has no meaning in establishing the copyright
it has not lost meaning in terms of licensing.
This change makes it so code under the Blender Foundation copyright does
not use "all rights reserved". This is also how the GPL license itself
states how to apply it to the source code:
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software ...
This change does not change copyright notice in cases when the copyright
is dual (BF and an author), or just an author of the code. It also does
mot change copyright which is inherited from NaN Holding BV as it needs
some further investigation about what is the proper way to handle it.
Certain material node graphs can be very expensive to run. This feature aims to produce secondary GPUPass shaders within a GPUMaterial which provide optimal runtime performance. Such optimizations include baking constant data into the shader source directly, allowing the compiler to propogate constants and perform aggressive optimization upfront.
As optimizations can result in reduction of shader editor and animation interactivity, optimized pass generation and compilation is deferred until all outstanding compilations have completed. Optimization is also delayed util a material has remained unmodified for a set period of time, to reduce excessive compilation. The original variant of the material shader is kept to maintain interactivity.
Also adding a new concept to gpu::Shader allowing assignment of a parent shader from which a shader can pull PSO descriptors and any required metadata for asynchronous shader cache warming. This enables fully asynchronous shader optimization, without runtime hitching, while also reducing runtime hitching for standard materials, by using PSO descriptors from default materials, ahead of rendering.
Further shader graph optimizations are likely also possible with this architecture. Certain scenes, such as Wanderer benefit significantly. Viewport performance for this scene is 2-3x faster on Apple-silicon based GPUs.
Authored by Apple: Michael Parkin-White
Ref T96261
Pull Request #104536
Compile each static shader using shaderc to Spir-V binaries.
The main goal is to make sure that the GLSL created using ShaderCreateInfo and able to compile to Spir-V.
For the second stage a correct pipeline needs to be created and some shader would need more
adjustments (push constants size).
With this patch future changes to GLSL sources can already be checked against vulkan, without the
backend finished.
Mechanism has been tested using MacOS and MoltenVK. For other OS, we should finetune CMake
files to find the right location to shaderc.
```
************************************************************
*** Build Mon 12 Dec 2022 11:08:07 CET
************************************************************
Shader Test compilation result: 463 / 463 passed (skipped 118 for compatibility reasons)
OpenGL backend shader compilation succeeded.
Shader Test compilation result: 529 / 529 passed (skipped 52 for compatibility reasons)
Vulkan backend shader compilation succeeded.
```
Reviewed By: fclem
Maniphest Tasks: T102760
Differential Revision: https://developer.blender.org/D16610
Required by Metal backend for efficient shader compilation. EEVEE material
resource binding permutations now controlled via CreateInfo and selected
based on material options. Other existing CreateInfo's also modified to
ensure explicitness for depth-writing mode. Other missing bindings also
addressed to ensure full compliance with the Metal backend.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D16243
To make GPU backends other than OpenGL work. Adds required pixel buffer and
fence objects to GPU module.
Authored by Apple: Michael Parkin-White
Ref T96261
Ref T92212
Reviewed By: fclem, brecht
Differential Revision: https://developer.blender.org/D16042
Full support for translation and compilation of shaders in Metal, using
GPUShaderCreateInfo. Includes render pipeline state creation and management,
enabling all standard GPU viewport rendering features in Metal.
Authored by Apple: Michael Parkin-White, Marco Giordano
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D15563
With the new `attrs_info_get` method, we can get information about
the attributes used in a `GPUShader` and thus have more freedom in the
automatic creation of `GPUVertFormat`s
Reviewed By: fclem, campbellbarton
Differential Revision: https://developer.blender.org/D15764
This is a complete rewrite of the draw debug drawing module in C++.
It uses `GPUStorageBuf` to store the data to be drawn and use indirect
drawing. This makes it easier to do a mirror API for GPU shaders.
The C++ API class is exposed through `draw_debug.hh` and should be used
when possible in new code.
However, the debug drawing will not work for platform not yet supporting
`GPUStorageBuf`. Also keep in mind that this module must only be used
in debug build for performance and compatibility reasons.
This patch implements the necessary changes to the GPU module that are
needed by the realtime compositor.
A new function GPU_material_from_callbacks was added to construct a GPU
material from a number of callbacks. A callback to construct the
material graph by adding and linking the necessary GPU material nodes.
And the existing code generator callback. This essentially allows the
construction of GPU materials independent of node trees and without the
need to do any node tree localization.
A new composite source output to the code generator was added. This
output contains the serialization of nodes that are tagged with
GPU_NODE_TAG_COMPOSITOR, which are the nodes linked to the newly added
composite output links.
Two new GPU uniform setters were added for int2 and matrix3 types.
Shader create info now supports generated compute sources.
Shaders starting with gpu_shader_compositor are now considered part of
the shader library.
Additionally, two fixes were implemented. First, GPU setter node
de-duplication now appropriately increments the reference count of the
references resources. Second, unlinked sockets now get their value from
their associated GPU node stack instead of the socket itself.
Differential Revision: https://developer.blender.org/D14690
Reviewed By: Clement
This commit removes all EEVEE specific code from the `gpu_shader_material*.glsl`
files. It defines a clear interface to evaluate the closure nodes leaving
more flexibility to the render engine.
Some of the long standing workaround are fixed:
- bump mapping support is no longer duplicating a lot of node and is instead
compiled into a function call.
- bump rewiring to Normal socket is no longer needed as we now use a global
`g_data.N` for that.
Closure sampling with upstread weight eval is now supported if the engine needs
it.
This also makes all the material GLSL sources use `GPUSource` for better
debugging experience. The `GPUFunction` parsing now happens in `GPUSource`
creation.
The whole `GPUCodegen` now uses the `ShaderCreateInfo` and is object type
agnostic. Is has also been rewritten in C++.
This patch changes a view behavior for EEVEE:
- Mix shader node factor imput is now clamped.
- Tangent Vector displacement behavior is now matching cycles.
- The chosen BSDF used for SSR might change.
- Hair shading may have very small changes on very large hairs when using hair
polygon stripes.
- ShaderToRGB node will remove any SSR and SSS form a shader.
- SSS radius input now is no longer a scaling factor but defines an average
radius. The SSS kernel "shape" (radii) are still defined by the socket default
values.
Appart from the listed changes no other regressions are expected.
Print it as a "%s" so that possible percentage symbols in the
error message does not cause issues.
Use proper assert (assert(true) is a no-op).
Also use `empty()` instead of `length()`.
Reviewed with Clement in real life.
In order to allow GLSL Cross Compilation across platforms, expose in
Python the `GPUShaderCreateInfo` strategy as detailed in
https://wiki.blender.org/wiki/EEVEE_%26_Viewport/GPU_Module/GLSL_Cross_Compilation
The new features can be listed as follows:
```
>>> gpu.types.GPUShaderCreateInfo.
define(
fragment_out(
fragment_source(
push_constant(
sampler(
typedef_source(
uniform_buf(
vertex_in(
vertex_out(
vertex_source(
>>> gpu.types.GPUStageInterfaceInfo.
flat(
name
no_perspective(
smooth(
>>> gpu.shader.create_from_info(
```
Reviewed By: fclem, campbellbarton
Differential Revision: https://developer.blender.org/D14497
Metal shading language follows the C++ 14 standard and in some cases requires a greater level of explicitness than GLSL. There are also some small language differences:
- Explicit type-casts (C++ requirements)
- Explicit constant values (C++ requirements, e.g. floating point values using 0.0 instead of 0).
- Metal/OpenGL compatibility paths
- GLSL Function prototypes
- Explicit accessors for vector types when sampling textures.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D14378
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Source files are now only referenced and listed for the driver to ingest.
Shader sources now includes generated data if any.
Also cleans up gpu_shader_dependency_get_builtins casts.
- Scan all static shaders for builtins on startup.
- Add possibility to manually add builtins.
- `ShaderCreateInfo.builtins_` contain builtins from all stages.
This patch migrates the draw manager hair refine compute shader to use
GPUShaderCreateInfo.
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D13915
This is a first part of the Shader Create Info system could be.
A shader create info provides a way to define shader structure, resources
and interfaces. This makes for a quick way to provide backend agnostic
binding informations while also making shader variations easy to declare.
- Clear source input (only one file). Cleans up the GPU api since we can create a
shader from one descriptor
- Resources and interfaces are generated by the backend (much simpler than parsing).
- Bindings are explicit from position in the array.
- GPUShaderInterface becomes a trivial translation of enums and string copy.
- No external dependency to third party lib.
- Cleaner code, less fragmentation of resources in several libs.
- Easy to modify / extend at runtime.
- no parser involve, very easy to code.
- Does not hold any data, can be static and kept on disc.
- Could hold precompiled bytecode for static shaders.
This also includes a new global dependency system.
GLSL shaders can include other sources by using #pragma BLENDER_REQUIRE(...).
This patch already migrated several builtin shaders. Other shaders should be migrated
one at a time, and could be done inside master.
There is a new compile directive `WITH_GPU_SHADER_BUILDER` this is an optional
directive for linting shaders to increase turn around time.
What is remaining:
- pyGPU API {T94975}
- Migration of other shaders. This could be a community effort.
Reviewed By: jbakker
Maniphest Tasks: T94975
Differential Revision: https://developer.blender.org/D13360
With the compute pipeline calculation can be offloaded to the GPU.
This patch only adds the framework for compute. So no changes for users at
this moment.
NOTE: As this is an OpenGL4.3 feature it must always have a fallback.
Use `GPU_compute_shader_support` to check if compute pipeline can be used.
Check `gpu_shader_compute*` test cases for usage.
This patch also adds support for shader storage buffer objects and device only
vertex/index buffers.
An alternative that had been discussed was adding this to the `GPUBatch`, this
was eventually not chosen as it would lead to more code when used as part of a
shading group. The idea is that we add an `eDRWCommandType` in the near
future.
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D10913
Resolves occasional glitch/flicker drawing dark buttons in the UI.
Regression in 405a5d3bd7
which removed shader unbinding when the batch is drawn.
GPU_shader_bind could run with the sRGB uniform in an unexpected state.
Reviewed By: fclem
Ref D11124
Modernize loops by using the `for(type variable : container)` syntax.
Some loops were trivial to fix, whereas others required more attention
to avoid semantic changes. I couldn't address all old-style loops, so
this commit doesn't enable the `modernize-loop-convert` rule.
Although Clang-Tidy's auto-fixer prefers to use `auto` for the loop
variable declaration, I made as many declarations as possible explicit.
To me this increases local readability, as you don't need to fully
understand the container in order to understand the loop variable type.
No functional changes.
