GPU_DATA_UINT_24_8 isn't used anymore. We cannot phase out the data type
as it can still be used by add-ons. This PR will deprecate
`GPU_DATA_UINT_24_8`. When used in an add-on a deprecation message will
be shown.
Pull Request: https://projects.blender.org/blender/blender/pulls/140715
Metal and AMD/Intel/Vulkan don't support depth24 texture formats
natively. The backends implemented fallback to use depth32f in
stead.
Recently we removed all usages of depth24 to use depth32 and the
next step is to remove the depth24 format and the workarounds in
the backend.
Note: The removal of `GPU_DATA_UINT_24_8` isn't part of this PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/140659
Blender uses depth24 for legacy reasons. All backends that we support
have support for depth32f.
This PR updates all usages of depth24 with depth32f.
- depth24 are not supported on AMD/Intel/Vulkan and Metal. There depth32f
was already used to work around this limitation.
- This allows us to implement reverse depth in workbench, overlay and
grease pencil in the future.
Pull Request: https://projects.blender.org/blender/blender/pulls/140531
The polyline workarounds were not working as expected
since #139627 as it was not garanteed that the polylines
shader would be correctly initialized with the workaround
tag.
Adding a wrapper class to ensure the initialization fixes
the issue.
This reduces the time needed to get to the first pixel
on screen by multithreading the builtin shader compilation.
We avoid doing this if subprocess compilation is on as
the overhead of potentially partially starting all subprocess
is far greater than the benefit of paralllel compilation.
For some reasons, the compilation is much slower when
done async for these shaders (on Metal ~200ms > ~1.2ms),
so the saving might not be substantial.
Mac M1: First frame 6s > 5s.
Pull Request: https://projects.blender.org/blender/blender/pulls/139627
The Extend Bounds input has no effect when the Fast Gaussian filter is
used. Similarly, it has no effect if the Bokeh Blur node is using
variable size. This is a known limitation and was just not implemented.
So to fix this, we implement a general solution that works globally
across the node by pre-padding the inputs of the blur. This uses more
memory but also speeds up the base case when Extend Bounds is disabled,
while also reducing the binary size due to fewer blur specializations.
The variable size Bokeh Blur test was updated since it Extend Bounds was
silently ignored.
Pull Request: https://projects.blender.org/blender/blender/pulls/140192
Prevent race conditions caused by calling `GPUWorker::wake_up` when the
worker is not waiting.
Found to be an issue in #139627, since `wake_up` is likely to be called
before the thread has fully started.
Pull Request: https://projects.blender.org/blender/blender/pulls/139842
This allows to generate source file that will
be injected in a predefined source dependance tree.
This allow much cleaner shader workflow where
all sources are explicitly referenced from the
main source file.
Pull Request: https://projects.blender.org/blender/blender/pulls/140047
This prevents the use of unaligned data types in
vertex formats. These formats are not supported on many
platform.
This simplify the `GPUVertexFormat` class a lot as
we do not need packing shenanigans anymore and just
compute the vertex stride.
The old enums are kept for progressive porting of the
backends and user code.
This will break compatibility with python addons.
TODO:
- [x] Deprecation warning for PyGPU (4.5)
- [x] Deprecate matrix attributes
- [x] Error handling for PyGPU (5.0)
- [x] Backends
- [x] Metal
- [x] OpenGL
- [x] Vulkan
Pull Request: https://projects.blender.org/blender/blender/pulls/138846
This works by wrapping the entry point call inside a
`main` function.
Since resources are still defined in global space,
function accessing these are marked with a custom
attribute. This custom attribute expands in a
`#ifdef` guard for the matching stage.
This is a temporary solution and will eventually
be lifted once we support SRD.
### TODO
- [ ] Implement `[[gpu::vertex/fragment_function]]`.
Pull Request: https://projects.blender.org/blender/blender/pulls/139233
This allows to reduce the waiting time caused by
shader compilation on some GPU-driver combo.
A new settings in the User Preferences make it
possible to override the default amount of worker
threads and optionally use subprocesses.
We still use only one worker thread in cases where
there is no benefit with adding more workers
(like AMD pro driver and Intel windows).
It doesn't scale as much as subprocesses for material
shader compilation but that is for other reasons
explained in #139818.
Add some heuristic to avoid too much memory usage
and / or too many stalls.
Also add some heuristic to the default number of subprocess for
the platform that shows scalling.
Historically, multithreaded compilation was prevented by the
need of context per thread inside `DRWShader` module.
Also there was no good scaling at that time. But
nowadays numbers shows different results with
good scaling with reasonable amount of threads on many
platforms.
Even if we are going for vulkan in the next release
most of the legacy hardware will still use OpenGL for
a few other releases. So it is relevant to make this
easy improvement.
See pull request for measurements.
Pull Request: https://projects.blender.org/blender/blender/pulls/139821
This has limited use cases since it doesn't
profile the heavy part of the vulkan backend.
Almost 1:1 port of the metal implementation from #139551.
Doesn't cover rendergraph submission nor GPU timings.
Pull Request: https://projects.blender.org/blender/blender/pulls/139899
When drawing image scope the vulkan backend raised some asserts. After
checking them the issue was in the calling code, that could lead to
undefined behavior on other platforms as well.
It isn't allowed to have an immediate mode shader bound, when performing
batch drawing. There was also a point batch that didn't use any point
shader resulting in undefined behavior as well.
For 5.0 we should add this as a GPU module check.
Pull Request: https://projects.blender.org/blender/blender/pulls/139926
Compilation constants are constants defined in the create info.
They cannot be changed after the shader is created.
It is a replacement to macros with added type safety.
Reuse most of the logic from Specialization constants.
Pull Request: https://projects.blender.org/blender/blender/pulls/139703
Allow adding compilation batches to different priority queues.
Set priorities so static shaders are always compiled first,
then materials, and optimized materials last.
Pull Request: https://projects.blender.org/blender/blender/pulls/139456
This PR adds builtin shaders for drawing points. Using `FLAT_COLOR`,
`SMOOTH_COLOR`, `UNIFORM_COLOR` can lead to undesired behavior
on Metal and Vulkan backends. To ensure future compatibility this PR
adds `POINT_FLAT_COLOR` and `POINT_UNIFORM_COLOR`.
The point size can be set using `gpu.state.point_size_set`.
Pull Request: https://projects.blender.org/blender/blender/pulls/139583
The GPU implementation is a bit too complex
to implement for now.
As we are improving shader loading, having the
CPU timings is already helpful.
Note that `Map<size_t, int>` does not compile
on Clang.
This is exposing the `--profile-gpu` option on
all backends as the vulkan backend should follow
shortly.
Pull Request: https://projects.blender.org/blender/blender/pulls/139551
This PR adds a clear on creation for draw region framebuffers, which
fixes issue https://projects.blender.org/blender/blender/issues/139443.
When using the "Lock Interface" option, redrawing is disabled until the
rendering process has finished. When the GPU driver does not initialize
framebuffer textures with zero, this means that visual corruption can
be perceived when resizing the main window while rendering.
This fix makes sure that framebuffer textures are initialized with zero
on all GPU platforms.
Pull Request: https://projects.blender.org/blender/blender/pulls/139445
This patch adds a new Image Coordinates node for the compositor. The
Texture Coordinates and Pixel Coordinates outputs were removed from the
Image Info node and were added to the new node instead. Additionally, a
new Normalized Coordinates output was added.
The Pixel Coordinates output now no longer includes half pixel offsets.
Pull Request: https://projects.blender.org/blender/blender/pulls/138935
Cleanup and simplification of GPUMaterial and GPUPass compilation.
See #133674 for details/goals.
- Remove the `draw_manage_shader` thread.
Deferred compilation is now handled by the gpu::ShaderCompiler
through the batch compilation API.
Batch management is handled by the `GPUPassCache`.
- Simplify `GPUMaterial` status tracking so it just queries the
`GPUPass` status.
- Split the `GPUPass` and the `GPUCodegen` code.
- Replaced the (broken) `GPU_material_recalc_flag_get` with the new
`GPU_pass_compilation_timestamp`.
- Add the `GPU_pass_cache_wait_for_all` and
`GPU_shader_batch_wait_for_all`, and remove the busy waits from
EEVEE.
- Remove many unused functions, properties, includes...
Pull Request: https://projects.blender.org/blender/blender/pulls/135637
Add back the deprecated format to avoid asserts.
GPU backends should still have the code to support them.
Also add deprecation warnings as these types will be
removed for 5.0.
Pull Request: https://projects.blender.org/blender/blender/pulls/139213
It can happen that the previous context drew with a different
colorspace. In the case where the new context is drawing with
the same shader that was previously bound (shader binding
optimization), the uniform would not be set again because the
dirty flag would not have been set (since the color space of
this new context never changed). The shader would reuse the same
colorspace as the previous context framebuffer (see #137855).
Fix#137855
Pull Request: https://projects.blender.org/blender/blender/pulls/139226
This caused UB in the tests now that tests are all ran
inside the same context.
A shader could be free but its pointer would be dangling
inside the `Context`. A new shader could have the same
address and generate UB after binding.
This is not the best way to solve this issue but at least
we prevent the use of the UB.
Pull Request: https://projects.blender.org/blender/blender/pulls/139109
This allows multiple threads to request different specializations without
locking usage of all specialized shaders program when a new specialization
is being compiled.
The specialization constants are bundled in a structure that is being
passed to the `Shader::bind()` method. The structure is owned by the
calling thread and only used by the `Shader::bind()`.
Only querying for the specialized shader (Map lookup) is locking the shader
usage.
The variant compilation is now also locking and ensured that
multiple thread trying to compile the same variant will never result
in race condition.
Note that this removes the `is_dirty` optimization. This can be added
back if this becomes a bottleneck in the future. Otherwise, the
performance impact is not noticeable.
Pull Request: https://projects.blender.org/blender/blender/pulls/136991
This feature greatly increase depth buffer precision.
This is very noticeable in large view distance scenes.
This is enabled by default on GPUs that supports it (most of the hardware we
support already supports this). This makes rendering different on the GPUs
that do not support that feature (`glClipControl`).
While this give much better depth precision as before, we also have a lot of
imprecision caused by our vertex transformations. This can be improved in
another task.
Pull Request: https://projects.blender.org/blender/blender/pulls/138898
This adds support for the extension and always
set the clip state value to 0..1 to align with vulkan
and metal. Moreover this is needed for the Reverse Z
implementation.
Note that this is a OpenGL 4.5 feature and is not
required to start Blender. So there must still be
a fallback path for now.
Rel #138898
Pull Request: https://projects.blender.org/blender/blender/pulls/138941
This commit finishes removing the uses of the integer to float
vertex buffer fetch mode. Previous commits noted below already started
that process. The last usage was geometry attributes. Now integers are
converted to floats as part of the existing upload process.
The change makes the Vulkan vertex buffer type conversion unused, so
it's removed. That's nice because Vulkan vertex buffers go from 1040 to
568 bytes in size and have significantly less overhead on creation.
Related:
- 153abc372e
- 1e1ac2bb9b
- 617858e453
Pull Request: https://projects.blender.org/blender/blender/pulls/138873
The conversion from int to float is not supported natively
so it ends up happening beforehand on the CPU or as a
step before the vertex buffer can be used. It's better to just
upload floats in the first place.
Related to:
- 1e1ac2bb9b
- 617858e453
Pull Request: https://projects.blender.org/blender/blender/pulls/138855
This unifies vertex and texture data formats
into a single base enum class.
`TextureFormat` and `VertexFormat` then mask
the invalid format for their respective usage.
Having a base enum allows casting between
`TextureFormat` and `VertexFormat` possible
(needed for Buffer Textures).
It also makes it easier to write and read data
to buffers/textures as each format will have an
associated host type.
These enum is generated from MACRO expansion.
This allow to centralize all information about
the formats in one place. This avoid duplicating
the list of enums for each backend.
This only creates the new enum. Porting older enums will
be done in other PRs.
Normalized integer CPU format are missing and waiting for #130640
Rel #130632
Pull Request: https://projects.blender.org/blender/blender/pulls/138069
Fix the recently implemented ShaderCompiler::batch_cancel.
Expose it with GPU_shader_batch_cancel and
GPU_shader_specialization_batch_cancel.
Use them in the EEVEE ShaderModule destructor, to prevent blocking on
destruction when there are in-flight compilations.
Pull Request: https://projects.blender.org/blender/blender/pulls/138774
Briefly about this change:
- OpenColorIO C-API is removed.
- The information about color spaces in ImBuf module is removed.
It was stored in global ListBase in colormanagement.cc.
- Both OpenColorIO and fallback implementation supports GPU drawing.
- Fallback implementation supports white point, RGB curves, etc.
- Removed check for support of GPU drawing in IMB.
Historically it was implemented in a separate library with C-API, this
is because way back C++ code needed to stay in intern. This causes all
sort of overheads, and even calls that are strictly considered bad
level.
This change moves OpenColorIO integration into a module within imbuf,
next to movie, and next to IMB_colormanagement which is the main user
of it. This allows to avoid copy of color spaces, displays, views etc
in the ImBuf: they were used to help quickly querying information to
be shown on the interface. With this change it can be stored in the
same data structures as what is used by the OpenColorIO integration.
While it might not be fully avoiding duplication it is now less, and
there is no need in the user code to maintain the copies.
In a lot of cases this change also avoids allocations done per access
to the OpenColorIO. For example, it is not needed anymore to allocate
image descriptor in a heap.
The bigger user-visible change is that the fallback implementation now
supports GLSL drawing, with the whole list of supported features, such
as curve mapping and white point. This should help simplifying code
which relies on color space conversion on GPU: there is no need to
figure out fallback solution in such cases. The only case when drawing
will not work is when there is some actual bug, or driver issue, and
shader has failed to compile.
The change avoids having an opaque type for color space, and instead
uses forward declaration. It is a bit verbose on declaration, but helps
avoiding unsafe type-casts. There are ways to solve this in the future,
like having a header for forward declaration, or to flatten the name
space a bit.
There should be no user-level changes under normal operation.
When building without OpenColorIO or the configuration has a typo or
is missing a fuller set of color management tools is applies (such as the
white point correction).
Pull Request: https://projects.blender.org/blender/blender/pulls/138433
Part of #136993.
Share as much of the ShaderCompiler implementations as possible.
Remove the ShaderCompiler/ShaderCompilerGeneric split and make most of
its functions non virtual.
Move the `get_compiler` function from `Context` to `GPUBackend` and
creation/deletion to `GPUBackend::init/delete_resources`.
Add a `batch_cancel` function to `ShaderCompiler` (needed for the
GPUPass refactor).
As a nice extra, the multithreaded OpenGL compilation has become faster
too.
The barbershop materials + EEVEE static shaders have gone from 27s to
22s.
I have not observed any performance difference on Vulkan or Metal.
Pull Request: https://projects.blender.org/blender/blender/pulls/136676
Multiple threads would be setting the globals
`g_shader_builtin_srgb_transform` and
`g_shader_builtin_srgb_is_dirty`.
These are use for color management inside the builtin
shaders. But the render thread could modify these
values even if its shader have no use of these.
The fix is to move these globals to the `gpu::Context`
class. This way we remove the race condition.
This patch adds a new `BLI_mutex.hh` header which adds `blender::Mutex` as alias
for either `tbb::mutex` or `std::mutex` depending on whether TBB is enabled.
Description copied from the patch:
```
/**
* blender::Mutex should be used as the default mutex in Blender. It implements a subset of the API
* of std::mutex but has overall better guaranteed properties. It can be used with RAII helpers
* like std::lock_guard. However, it is not compatible with e.g. std::condition_variable. So one
* still has to use std::mutex for that case.
*
* The mutex provided by TBB has these properties:
* - It's as fast as a spin-lock in the non-contended case, i.e. when no other thread is trying to
* lock the mutex at the same time.
* - In the contended case, it spins a couple of times but then blocks to avoid draining system
* resources by spinning for a long time.
* - It's only 1 byte large, compared to e.g. 40 bytes when using the std::mutex of GCC. This makes
* it more feasible to have many smaller mutexes which can improve scalability of algorithms
* compared to using fewer larger mutexes. Also it just reduces "memory slop" across Blender.
* - It is *not* a fair mutex, i.e. it's not guaranteed that a thread will ever be able to lock the
* mutex when there are always more than one threads that try to lock it. In the majority of
* cases, using a fair mutex just causes extra overhead without any benefit. std::mutex is not
* guaranteed to be fair either.
*/
```
The performance benchmark suggests that the impact is negilible in almost
all cases. The only benchmarks that show interesting behavior are the once
testing foreach zones in Geometry Nodes. These tests are explicitly testing
overhead, which I still have to reduce over time. So it's not unexpected that
changing the mutex has an impact there. What's interesting is that on macos the
performance improves a lot while on linux it gets worse. Since that overhead
should eventually be removed almost entirely, I don't really consider that
blocking.
Links:
* Documentation of different mutex flavors in TBB:
https://www.intel.com/content/www/us/en/docs/onetbb/developer-guide-api-reference/2021-12/mutex-flavors.html
* Older implementation of a similar mutex by me:
https://archive.blender.org/developer/differential/0016/0016711/index.html
* Interesting read regarding how a mutex can be this small:
https://webkit.org/blog/6161/locking-in-webkit/
Pull Request: https://projects.blender.org/blender/blender/pulls/138370
Currently unused, but allows areas outside of DRW to render to the
color render and depth texture.
The primary user of this new API will be Sequencer preview to draw
HDR images.
Ref #138094
Pull Request: https://projects.blender.org/blender/blender/pulls/138306
