WITH_VULKAN_GUARDEDALLOC is a development option to use Blenders guarded
allocator when allocating internal vulkan driver resources. It does not provide any benefits
as this should be covered by vulkan validation and drivers are often ignoring this. This
change will remove the option from cmake and source code.
Pull Request: https://projects.blender.org/blender/blender/pulls/129039
Intel Windows drivers for 10th gen and lower has some strange behavior
when using dynamic rendering. It requires pipeline conditions to be met,
when beginning a new rendering scope. This is strange as specs + VVL
notes that these conditions should be met during vkCmdDraw* commands.
Pull Request: https://projects.blender.org/blender/blender/pulls/129055
This change makes unused attachments extension optional.
This extension is fairly new and not all drivers have support for it.
The workaround will create additional pipelines when attachments are
not set.
Pull Request: https://projects.blender.org/blender/blender/pulls/129046
VK_EXT_dynamic_rendering_unused_attachments is required for correct working.
Renderdoc hides this extension, but most platforms do work. However the
Windows Intel driver crashes when using iGPUs; they don't support this
extension at all.
This change does a more strict extension test so drivers that do not
support this extension will fallback to OpenGL. When using renderdoc it
is now allowed to compile blender with `WITH_RENDERDOC=On`.
Future developments are needed to add support for Intel iGPUs on
Windows.
Pull Request: https://projects.blender.org/blender/blender/pulls/128986
Resources are shared, when running multiple contexts on the same thread.
Cycles uses the same context on multiple threads and expected same resources.
This change will introduce a single render graph per context and an updated
resource management. Render graphs are not shared anymore; Resource pools
are still shared, but garbage collection depends on the thread and if
background rendering is used.
Pull Request: https://projects.blender.org/blender/blender/pulls/128983
Cycles uses multiple threads to send commands to the GPU. The current
command buffer structure assumed that all commands from the same context
were send via the same thread. This wasn't the case and could lead to
recording commands to command buffers that are still pending (preparing
commands to send to GPU).
This is fixed by creating a command buffer each time a render graph
submits its work.
Detected when researching #128608
Pull Request: https://projects.blender.org/blender/blender/pulls/128978
When allocating a host visible buffer it could be that the returned
buffer was not host visible and access to the buffer would write to
unallocated memory.
Detected when researching #128608
Pull Request: https://projects.blender.org/blender/blender/pulls/128977
When using cycles in the viewport there it uses render threads and
workers to update the viewport. All threads can record commands to the
queue and needs to be synchronized. This didn't happen leading to
incorrect renders.
Detected when researching #128608
Pull Request: https://projects.blender.org/blender/blender/pulls/128975
Multiple threads can access the same device queue from different
threads. This could happen when doing a cycles preview render, baking
eevee volume probes or generating material previews.
This PR adds a mutex around access to the device queues.
Detected when researching #128608
Pull Request: https://projects.blender.org/blender/blender/pulls/128974
Adds an additional precheck to identify if the app cache dir is correct.
Reduces placing cache files all over the place when the app dir isn't
correct.
Adds a SPIR-V cache that skips frontend compilation for shaders
that are already compiled in a previous run of Blender.
Initially this was postponed to 4.4 but it was observed that
the vulkan backend didn't perform well on Windows in debug
builds. The reason is that the compiler would also be a debug
build which makes compiling a shader really slow. Starting
Blender on a debug build could take minutes.
So the decision was made to give this task a higher priority so
the vulkan backend would become more usable to developers
as well.
The cache is stored in the application cache dir. The SPIR-V
binaries can be used by different Blender versions so there
is no version specific cache folder.
**Sidecar**: SPIR-V files are a stream of bytes. There is no
header information that allow us to validate the stream. To
add basic validations we could add our custom header or
a sidecar. It was chosen to use a sidecar as having the SPIR-V
files unmodified allows us to load them directly in
debug tools for analyzing.
**Retention**: Shaders that are not used are automatically
removed with a retention period of 30 days.
**Shader builder**: Shader builder cannot use the SPIR-V
cache as it uses stubs that returns invalid cache directories.
This would load/save the cache to the location where you
started the build.
Pull Request: https://projects.blender.org/blender/blender/pulls/128741
Move most of the string preprocessing used for MSL
compatibility to `glsl_preprocess`.
Enforce some changes like matrix constructor and
array constructor to the GLSL codebase. This is
for C++ compatibility.
Additionally reduce the amount of code duplication
inside the compatibility code.
Pull Request: https://projects.blender.org/blender/blender/pulls/128634
When attaching a layered image with offset, the size of the attached
layers should be decreased. Otherwise an image view is created that can
access incorrect data.
Pull Request: https://projects.blender.org/blender/blender/pulls/128583
Related to #126974, which removed command reordering due to some
EEVEE/framebuffer requirements. However buffer can still be reordered
without any artifacts.
Update buffers are common operations and are often isolated; safe to
move them outside the rendering scope.
Pull Request: https://projects.blender.org/blender/blender/pulls/128538
When using AMD pro-drivers the limits reported of the device can be
`UINT_MAX` but are stored in int fields. In this case the limits would
become negative and GPU materials validation failed resulting into errors.
This is fixed by clamping the value to `INT_MAX`.
Pull Request: https://projects.blender.org/blender/blender/pulls/128437
This change adds the option to update a buffer via the render
graph via `vkCmdUpdateBuffer`. This is only enabled for
uniform buffers as they are small and aligned/sized correctly.
Pull Request: https://projects.blender.org/blender/blender/pulls/128416
This PR introduces support for the extension `VK_KHR_fragment_shader_barycentric`,
and includes a few miscellaneous improvements related to it.
1. Add support for `VK_KHR_fragment_shader_barycentric`, if the physical device
supports it. Otherwise, gpu_BaryCoord is generated through an injected geom
shader, like it was previously.
2. Simplify the logic of checking has_geometry_stage in vert shader.
3. Fix a potential issue of location mismatch in an injected geom shader.
Related to #127687Resolves#126228
Pull Request: https://projects.blender.org/blender/blender/pulls/127995
When performing preview job rendering the memory wasn't recycled leading
to a memory leak. For background rendering we already recycled memory in
a correct way. This change enables the same branch during preview
rendering.
Also adds a better `VKDevice::debug_print` to see the resources being
tracked by the different threads and resource pools.
Pull Request: https://projects.blender.org/blender/blender/pulls/128377
Pipeline pool could log to much information that confused developers who
are not up to date what pipelines are. This PR will hide the confusing
messages. When working on Vulkan these messages can still be shown by
raising the log level.
See !128254
Pull Request: https://projects.blender.org/blender/blender/pulls/128352
When exiting the immediate buffers are discarded, but where not
destroyed making the buffers still leak.
Detected when looking into descriptor set freeze issue.
Pull Request: https://projects.blender.org/blender/blender/pulls/128249
A driver (package) installed by the user can have many different drivers
and they can all report a different version. For AMD the version we
reported was from their Vulkan driver. This version isn't useful during
bug triaging.
This PR will use the driver info and driver name from the driver
properties to construct a driver version string that will be used for
reporting.
Pull Request: https://projects.blender.org/blender/blender/pulls/128232
Immediate mode uses the old 'resource tracker' which has been replaced
by swap chain resource pools. This PR optimizes immediate mode buffers
by utilizing resource pools.
Pull Request: https://projects.blender.org/blender/blender/pulls/128188
Currently the log only contained the first compatible device. It is
more important to the user to know which device is used.
This PR increases the level of the first compatible device so it is only
visible when increasing the log level. It reports the device, driver and
vendor when starting blender with `--debug-gpu`.
Pull Request: https://projects.blender.org/blender/blender/pulls/128168
Resource binding was over-complicated as I didn't understood the state
manager and vulkan to make the correct decisions at that time. This
refactor will remove a lot of the complexity and improves the performance.
**Performance**
The performance improvement is noticeable in complex grease pencil
scenes.
Grease pencil benchmark file picknick:
- `NVIDIA Quadro RTX 6000` 17 fps -> 24 fps
- `Intel(R) Arc(tm) A750 Graphics (DG2)` 6 -> 21 fps
**Bottle-neck**
The performance improvements originates from moving the update entry
point from state manager to shader interface. The previous implementation
(state manager) had to loop over all the bound resources and find in the
shader interface where it was located in the descriptor set. Ignoring
resources that were not used by the shader. But also making it hard
to determine if descriptor sets actually changed. Previous implementation
assumed descriptor sets always changed.
When descriptor set changed a new descriptor set needed to be allocated.
Most drivers this is a fast operation, but on Intel/Mesa this was measurable
slow. Using an allocation pool doesn't fit the Vulkan API as you are only
able to reuse when the layout matches exactly. Of course doable, but requires
another structure to keep track of the actual layouts.
**Solution**
By using the shader interface as entry point we can:
1. Keep track if there are any changes in the state manager. If not and the
layout is the same, the previous shader can be reused.
2. In stead of looping over each bound resource, we loop over bind points.
**Future extensions**
Bundle all descriptor set uploads just before use. This would be more
in line with how 'modern' Vulkan should be implemented. This PR already
separates the uploading from the updating and technically allows to upload
more than one descriptor set.
Instead of looking 1 set back we should measure if we can handle multiple
or keep track of the different layouts resources to improve the performance
even further.
Optional use `VK_KHR_descriptor_buffer` when available.
Pull Request: https://projects.blender.org/blender/blender/pulls/128068
GPU resources created during Light probe bake job were added to discard pool, but the pool itself was never notified by worker thread to release resources.
Bake job creates dedicated `GPUContext` for its needs and later deletes it within the same thread.
Pull Request: https://projects.blender.org/blender/blender/pulls/127977
Removes two levels of indirection when updating descriptor sets.
These are the easy ones to remove. Others will be removed in
a future PR.
This is part of reworking of how descriptor sets are used.
This PR Mostly reduces complexity.
Pull Request: https://projects.blender.org/blender/blender/pulls/127915
De-interleaved vertex buffers offsets the attribute in the buffer to
the de-interleaved position. The vertex attribute offset is limited by a
constrained and would raise an error when the buffers just a bit larger.
*VUID-VkVertexInputAttributeDescription-offset-00622*: offset must
be less than or equal to `VkPhysicalDeviceLimits::maxVertexInputAttributeOffset`
This PR fixes this by offsetting the buffer in stead of the attribute.
Offsetting buffers is limited by the amount of memory.
Pull Request: https://projects.blender.org/blender/blender/pulls/128031
Allows users to override the auto detection for GPU
selection. Normally the GPU selection is done by looping
over the order Vulkan provides and finding the highest
performing device based on its type (discrete, integrated,
software).
However users might have multiple discrete cards and want
to switch between them. Or developers want to validate other
GPUs without rebooting.
This PR adds the ability to override the auto detection
for the vulkan backend.
**Future improvements**:
- This PR does not include a command line option. This can be added
later for render farms.
Pull Request: https://projects.blender.org/blender/blender/pulls/127860
Since parallel compilations was introduced, a validation error
was signalling that push constants for compute shaders didn't have
the correct pipeline binding. The root cause was that the pipeline
binding was determined, before the type of shader was known.
This PR fixes this by detemining if a shader is a compute shader up
front. It also removes some code that could lead to issues.
Pull Request: https://projects.blender.org/blender/blender/pulls/128010
In addition to float<->half functions to convert one number (#127708), add
float_to_half_array and half_to_float_array functions:
- On x64, this uses SSE2 4-wide implementation to do the conversion
(2x faster half->float, 4x faster float->half compared to scalar),
- There's also an AVX2 codepath that uses CPU hardware F16C instructions
(8-wide), to be used when/if blender codebase will start to be built
for AVX2 (today it is not yet).
- On arm64, this uses NEON VCVT instructions to do the conversion.
Use these functions in Vulkan buffer/texture conversion code. Time taken to
convert float->half texture while viewing EXR file in image space (22M
numbers to convert): 39.7ms -> 10.1ms (would be 6.9ms if building for AVX2)
Pull Request: https://projects.blender.org/blender/blender/pulls/127838
This PR introduces parallel shader compilation for Vulkan shader
modules. This will improve shader compilation when switching to material
preview or EEVEE render preview. It also improves material compilation.
However in order to measure the differences shaderc needs to be updated.
PR has been created so we can already start with the code review. This
PR doesn't include SPIR-V caching, what will land in a separate PR as
it needs more validation.
Parallel shader compilation has been tested on AMD/NVIDIA on Linux.
Testing on other platforms is planned in the upcoming days.
**Performance**
```
AMD Ryzen™ 9 7950X × 32, 64GB Ram
Operating system: Linux-6.8.0-44-generic-x86_64-with-glibc2.39 64 Bits, X11 UI
Graphics card: Quadro RTX 6000/PCIe/SSE2 NVIDIA Corporation 4.6.0 NVIDIA 550.107.02
```
*Test*: Start blender, open barbershop_interior.blend and wait until the viewport
has fully settled.
| Backend | Test | Duration |
| ------- | ------------------------- | -------- |
| OpenGL | Coldstart/No subprocesses | 1:52 |
| OpenGL | Coldstart/8 Subprocesses | 0:54 |
| OpenGL | Warmstart/8 Subprocesses | 0:06 |
| Vulkan | Coldstart Without PR | 0:59 |
| Vulkan | Warmstart Without PR | 0:58 |
| Vulkan | Coldstart With PR | 0:33 |
| Vulkan | Warmstart With PR | 0:08 |
The difference in time (why OpenGL is faster in a warm start is that all
shaders are cached). Vulkan in this case doesn't cache anything and all
shaders are recompiled each time. Caching the shaders will be part of
a future PR. Main reason not to add it to this PR directly is that SPIR-V
cannot easily be validated and would require a sidecar to keep SPIR-V
compatible with external tools..
**NOTE**:
- This PR was extracted from #127418
- This PR requires #127564 to land and libraries to update. Linux lib
is available as attachment in this PR. It works without, but is as slow as
single threaded compilation.
Pull Request: https://projects.blender.org/blender/blender/pulls/127698
Windows/Intel and Apple drivers do not support dynamic
rendering unused attachments. Due to mistakes we made
this extension partly optional. Eg. the extension was
optional, but its settings were not.
This PR makes the extension fully optional. However
without the extension some drivers might make incorrect
assumptions. This should be solved when it is more clear
why some drivers are still crashing when using dynamic
rendering.
Pull Request: https://projects.blender.org/blender/blender/pulls/127839
Blender codebase had two ways to convert half (FP16) to float (FP32):
- BLI_math_bits.h half_to_float. Out of 64k possible half values, it converts
4096 of them incorrectly. Mostly denormals and NaNs, which is perhaps not too
relevant. But more importantly, it converts half zero to float 0.000030517578
which does not sound ideal.
- Functions in Vulkan vk_data_conversion.hh. This one converts 2046 possible
half values incorrectly.
Function to convert float (FP32) to half (FP16) was in Vulkan
vk_data_conversion.hh, and it got a bunch of possible inputs wrong. I guess it
did not do proper "round to nearest even" that CPU/GPU hardware does.
This PR:
- Adds BLI_math_half.hh with float_to_half and half_to_float functions.
- Documentation and test coverage.
- When compiling on ARM NEON, use hardware VCVT instructions.
- Removes the incorrect half_to_float from BLI_math_bits.h and replaces single
usage of it in View3D color picking to use the new function.
- Changes Vulkan FP32<->FP16 conversion code to use the new functions, to fix
correctness issues (makes eevee_next_bsdf_vulkan test pass). This makes it
faster too.
Pull Request: https://projects.blender.org/blender/blender/pulls/127708
ShaderC compiler was cached on the Vulkan backend. The compiler itself
is light-weight and doesn't require any caching. This PR removes the
cached instance from the backend.
Pull Request: https://projects.blender.org/blender/blender/pulls/127693
This uses the path that metal was using.
This doesn't seems to create any difference in render
tests. This simplify the backend code and avoid
specific path for metal.
Idea suggested by Kevin Chuang
Pull Request: https://projects.blender.org/blender/blender/pulls/127687
Parallel shader compilation introduced `GPU_shader_cache_dir_clear_old`.
The implementation was specific to OpenGL and could not be overwritten
by other backends. This PR improves the implementation so the backend
can have its own implementation.
This is needed for upcoming changes to the Vulkan backend where we
want to use similar mechanisms to speed up shader compilation and caching.
Pull Request: https://projects.blender.org/blender/blender/pulls/127680
