The current Vulkan resource management has some issues as context that
are not active can still use resources that are freed via another
context.
When this happens incorrect data can be read on the GPU and even crash
Blender. When trying to bind something that now contains other memory
pointers.
This change introduces that contexts are tracked via the device.
Context will be registered/unregistered with the device instance.
Unbinding of resources must pass the device and the device will check
all registered contexts. Binding of resources will happen via the active
context only.
On user perspective this now allowes:
- Opening/switching files
- Switching workspaces
- Switching render engines
Pull Request: https://projects.blender.org/blender/blender/pulls/108968
This PR adds support for indirect compute.
Indirect compute is almost the same as regular compute. The
only difference is that the parameters for the compute dispatch
isn't passed as a parameter, but that these parameters are part
of a buffer.
Pull Request: https://projects.blender.org/blender/blender/pulls/108879
This PR adds initial cube (array) support. Depending on how the texture
is used a different image view is created. When used as a framebuffer
attachment only a single side of the cubemap is attached. The image
view is attached as a 2d texture array. When used as a shader resource
the image view is a cubemap.
Also adds test cases to test both scenarios.
Pull Request: https://projects.blender.org/blender/blender/pulls/108794
* opengl_context -> system_gpu_context. This is the operating system OpenGL,
Metal or Vulkan context provided by GHOST.
* gpu_context -> blender_gpu_context. This is the GPUContext provided by
the Blender GPU module, which wraps the GHOST context and adds some state.
* Various functions create/destroy/enable/disable both contexts, these have
just gpu_context in the name now.
Pull Request: https://projects.blender.org/blender/blender/pulls/108723
Adds support to create SRGBA8 textures using UBYTE as input data.
It is assumed that the UBYTE are already converted to SRGB and no
conversion actually needs to happen.
Pull Request: https://projects.blender.org/blender/blender/pulls/108741
Some vulkan platforms don't support 24bit depth components. In this
case we will allocate float depth component. During readback the data
should be converted back to what is expected.
Pull Request: https://projects.blender.org/blender/blender/pulls/108698
This PR adds conversion template to convert between Low Precision float
formats. These include Binary32 floats and lower. It also adds support
to convert between unsigned and signed float formats and float formats
with different mantissa and exponents.
Additionally overflows (values that don't fit in the target float
format) will be clamped to the maximum value.
**Reasoning**:
Up to now the Vulkan backend only supported float and half float
formats, but to support workbench, 11 and 10 unsigned floats have to be
supported as well. The available libraries that support those float
formats targets scientific applications. Where the final code couldn't
be optimized that well by the compiler.
Data conversion for color pixels have different requirements about
clamping and sign, what could eliminate some clamping code in other
areas in Blender as well. Also could fix some undesired overflow when
using pixels with high intensity that didn't fit in the texture format
leading to known artifects in Eevee and slow-down in the image editor.
**Future**
In the future we might want to move this to the public part of the GPU
module so we can use this as well in other areas (Metal backend), Imbuf clamping
See 3c658d2c2e69e9cf97dfaa7a3c164262aefb9e76 for a commit that uses
this and improves image editor massively as it doesn't need to reiterate over
the image buffer to clamp the values into a known range.
Pull Request: https://projects.blender.org/blender/blender/pulls/108168
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
Vulkan doesn't have a conversion from uint32_t/int32_t to float. It does
have conversions from 16/8 bits. Main reason is that Vulkan expects that
there is no benefit when converting 32 bits from one type to the other
and should be solved by passing the right data type.
In Blender however this isn't the case as there are benefits on other
GPU backends (OpenGL for example).
This PR adds helper function to check if conversion is needed and
perform any conversions in place. It also implements the function to
upload vertex buffers to the GPU.
NOTE: Test cases have been added to validate this, but they are not
able to run on the Vulkan backend just yet, because they require the
graphics pipeline to be available.
Pull Request: https://projects.blender.org/blender/blender/pulls/107733
This PR adds several tests to GTest for testing blend states
and immediate mode. The immediate mode test are basic and will
be extended in the near future.
The tests have been developed in order to get the first pixel on
screen for the Vulkan backend. In the first phase of this goal we
had to validate that everything was working as expected without
being able to validate it by starting Blender.
Pull Request: https://projects.blender.org/blender/blender/pulls/107834
Vulkan test cases uses SRGB conversion. These test cases where failing
as the conversion table wasn't initialized.
This PR initializes the conversion table and fix the Vulkan data
conversion tests.
This PR uses the VK_EXT_debug_utils extension, but it's only for labeling, so it doesn't rely on the VK_LAYER_KHRONOS_validation functionality.
The functions that do these things are loaded into the runtime as vulkan extensions.
Declare the function pointers in a struct and make them members of vk_context.
Pull Request: https://projects.blender.org/blender/blender/pulls/106098
The renderdoc integration used to be behind the `--debug-gpu`
command line option. When using `--debug-gpu` outside renderdoc
error messages where displayed that aren't relevant.
This PR adds a specific command line option for the renderdoc
integration. This option will also enable `--debug-gpu`.
Pull Request: https://projects.blender.org/blender/blender/pulls/106541
For example
```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```
becomes
```
OIIOOutputDriver::~OIIOOutputDriver() {}
```
Saves quite some vertical space, which is especially handy for
constructors.
Pull Request: https://projects.blender.org/blender/blender/pulls/105594
This PR adds support for clearing framebuffers and scissor testing.
Tweaks had to be made to VKTexture to keep track of its layout
on the device. Based on the actual call the layout can switch
to a more optimum layout.
For example during upload of a texture the texture will be converted
to a transfer destination optimized layout. When reading from the
texture it will be converted to a transfer source optimized layout.
The order of the attachments in the framebuffer follows the next rules
- When only color attachments are there the color attachments will
be placed in the slot they are defined. This way it will match
the ShaderCreateInfo binding location.
- When a stencil/depth attachment is added it will be placed
right after the color attachments. When there isn't a color
attachment it will be the first attachment.
Pull Request: https://projects.blender.org/blender/blender/pulls/106044
This PR adds basic support for texture update, read back and clearing
for Vulkan. In Vulkan we need to convert each data type ourselves as
vulkan buffers are untyped. Therefore this change mostly is about data
conversions.
Considerations:
- Use a compute shader to do the conversions:
- Leads to performance regression as compute pipeline can stall
graphics pipeline
- Lead to additional memory usage as two staging buffers are needed
one to hold the CPU data, and one to hold the converted data.
- Do inline conversion when sending the data to Vulkan using `eGPUDataFormat`
- Additional CPU cycles required and not easy to optimize as it the
implementation requires many branches.
- Do inline conversion when sending the data to Vulkan (optimized for CPU)
For this solution it was chosen to implement the 3rd option as it is fast
and doesn't require additional memory what the other options do.
**Use Imath/half.h**
This patch uses `Imath/half.h` (dependency of OpenEXR) similar to
alembic. But this makes vulkan dependent of the availability of
OpenEXR. For now this isn't checked, but when we are closer to
a working Vulkan backend we have to make a decision how to cope with
this dependency.
**Missing Features**
*Framebuffer textures*
This doesn't include all possible data transformations. Some of those
transformation can only be tested after the VKFramebuffer has been
implemented. Some texture types are only available when created for a
framebuffer. These include the depth and stencil variations.
*Component format*
Is more relevant when implementing VKVertexBuffer.
*SRGB textures*
SRGB encoded textures aren't natively supported on all platforms, in
all usages and might require workarounds. This should be done in a
separate PR in a later stage when we are required to use SRGB textures.
**Test cases**
The added test cases gives an overview of the missing bits and pieces of
the patch. When the implementation/direction is accepted more test cases
can be enabled/implemented.
Some of these test cases will skip depending on the actual support of
platform the tests are running on. For example OpenGL/NVidia will skip
the next test as it doesn't support the texture format on OpenGL, although
it does support it on Vulkan.
```
[ RUN ] GPUOpenGLTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
[ SKIPPED ] GPUOpenGLTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI [ RUN ] GPUVulkanTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
[ OK ] GPUVulkanTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
```
Pull Request: https://projects.blender.org/blender/blender/pulls/105762
In Vulkan multiple commands can be in flight simultaneously.
These commands can share resources like descriptor sets or push
constants. When between commands these resources are updated
a new version of the resources should be created.
When a resource is updated it should check the submission id of the
command buffer. If this is different than last known by the resources,
the previous resources should be freed.
If the submission id is the same than previously it has to create a
new version of the resource to not intervene with other commands that
uses the resource before the update.
When the resource wasn't updated between multiple usages in the same
submission id it could reuse the previous resource.
This PR introduces a `ResourceTracker` and a `SubmissionTracker`.
A submission tracker can check if the command buffer is submitted.
In this case all resources of the resource tracker should be freed.
Unmodified resources in the same submission can be shared.
A resource tracker will keep track of all resources that are in
flight. After the resources are used (submission + execution) have
finished the resources can be cleared.
Pull Request: https://projects.blender.org/blender/blender/pulls/105183
This PR uses renderdoc for frame capturing when enabled.
It enabled an easier workflow for frame capturing.
- Capture GPU API calls from test cases
- Capture GPU API calls from background threads
- Capture GPU API calls from background rendering.
Renderdoc is an important GPU debugger used by the Eevee/
Viewport module. Previously we needed to change code in
order to record background rendering, that could on its own
lead to other side-effects.
The integration with renderdoc can be enabled using
`WITH_RENDERDOC=On` compiler option. `GPU_debug_capture_begin`
and `GPU_debug_capture_end` can be added to the section
of the code you want to debug. When running Blender inside
renderdoc this part will automatically be captured.
All GPU test cases are now guarded by these calls. In order
to capture the test cases you need to start the test cases
from renderdoc and the captured GPU API calls will appear
where each capture is a single test case.
Pull Request: https://projects.blender.org/blender/blender/pulls/105921
The _gpu_ prefix is redundant as they are inside a namespace
and run as part of a test suite that already contain the name
gpu. (GPUOpenGLTest)
This patch also moved the texture test cases to its own
compile module.
When a shader is bound it should outlife the pipeline. In one
test case where only the shader data was accessed it also bound
the shader. This isn't needed as the shader data should be
retrieved without binding the shader.
This change fixes the issue by not binding the GPU shader.
Both the shader_builder and existing shader tests eventually
tested the same aspects. shader_builder is more modern and
handles more cases.
The old shader test requires a full backend in order to run
This commit replaces the old tests to just use the
shader builder for validation.
Shader builder can still be run at compile time, this is
just a convenience to have as a test case as well for CI/CD.
Ref: #105482
**What are push constants?**
Push constants is a way to quickly provide a small amount of uniform data to shaders.
It should be much quicker than UBOs but a huge limitation is the size of data - spec
requires 128 bytes to be available for a push constant range.
**What are the challenges with push constants?**
The challenge with push constants is that the limited available size. According to
the Vulkan spec each platform should at least have 128 bytes reserved for push
constants. Current Mesa/AMD drivers supports 256 bytes, but Mesa/Intel is only 128
bytes.
**What is our solution?**
Some shaders of Blender uses more than these boundaries. When more data is needed
push constants will not be used, but the shader will be patched to use an uniform
buffer instead. This mechanism will be part of the Vulkan backend and shader
developers should not see any difference on API level.
**Known limitations**
Current state of the vulkan backend does not track resources that are in the
command queue. This patch includes some test cases that identified this issue as
well. See #104771.
Pull Request #104880
Resolves issue with nearest filtering on UI Icons. Note that as
Metal does not support LOD bias as a parameter on a sampler
object, the original code has been modified to perform LOD
biasing at the shader level.
As GPU_SAMPLER_ICON is not widely used, it is more
efficient to apply directly to the affected shaders, rather
than workaround passing in the sampler LOD bias as a
separate value e.g. uniform or push constant.
Original PR feedback addressed to also refactor ICON
shaders to use consistent style for single and multi
Icon rendering.
Authored by Apple: Michael Parkin-White
Ref #96261
Pull Request #105145
Descriptor set locations are now determined in the
VKShaderInterface. Issues with the previous solution:
- Due to legacy code in GPU module the locations/bindings
must be the same. Using one for something else might
result in undesired lookups, incorrect resource
bindings.
- Images/Textures reuses the same namespace, that didn't
work as expected when looking up the resources via
its binding.
This refactoring is required for adding support for
push constants.
Pull Request #105073
This patch adds initial support for compute shaders to
the vulkan backend. As the development is oriented to the test-
cases we have the implementation is limited to what is used there.
It has been validated that with this patch that the following test
cases are running as expected
- `GPUVulkanTest.gpu_shader_compute_vbo`
- `GPUVulkanTest.gpu_shader_compute_ibo`
- `GPUVulkanTest.gpu_shader_compute_ssbo`
- `GPUVulkanTest.gpu_storage_buffer_create_update_read`
- `GPUVulkanTest.gpu_shader_compute_2d`
This patch includes:
- Allocating VkBuffer on device.
- Uploading data from CPU to VkBuffer.
- Binding VkBuffer as SSBO to a compute shader.
- Execute compute shader and altering VkBuffer.
- Download the VkBuffer to CPU ram.
- Validate that it worked.
- Use device only vertex buffer as SSBO
- Use device only index buffer as SSBO
- Use device only image buffers
GHOST API has been changed as the original design was created before
we even had support for compute shaders in blender. The function
`GHOST_getVulkanBackbuffer` has been separated to retrieve the command
buffer without a backbuffer (`GHOST_getVulkanCommandBuffer`). In order
to do correct command buffer processing we needed access to the queue
owned by GHOST. This is returned as part of the `GHOST_getVulkanHandles`
function.
Open topics (not considered part of this patch)
- Memory barriers & command buffer encoding
- Indirect compute dispatching
- Rest of the test cases
- Data conversions when requested data format is different than on device.
- GPUVulkanTest.gpu_shader_compute_1d is supported on AMD devices.
NVIDIA doesn't seem to support 1d textures.
Pull-request: #104518
This replaces `GPU_SHADER_3D_POINT_FIXED_SIZE_VARYING_COLOR` by
GPU_SHADER_2D_POINT_UNIFORM_SIZE_UNIFORM_COLOR_OUTLINE_AA`.
None of the usage made sense to not use the AA shader.
Scale the point size to account for the rounded shape.
The GPU module has 2 different styles when reading back data from
GPU buffers. The SSBOs used a memcpy to copy the data to a
pre-allocated buffer. IndexBuf/VertBuf gave back a driver/platform
controlled pointer to the memory.
Readback is done for test cases returning mapped pointers is not safe.
For this reason we settled on using the same approach as the SSBO.
Copy the data to a caller pre-allocated buffer.
Reason why this API is currently changed is that the Vulkan API is more
strict on mapping/unmapping buffers that can lead to potential issues
down the road.
Pull Request #104571
Implements virtual shadow mapping for EEVEE-Next primary shadow solution.
This technique aims to deliver really high precision shadowing for many
lights while keeping a relatively low cost.
The technique works by splitting each shadows in tiles that are only
allocated & updated on demand by visible surfaces and volumes.
Local lights use cubemap projection with mipmap level of detail to adapt
the resolution to the receiver distance.
Sun lights use clipmap distribution or cascade distribution (depending on
which is better) for selecting the level of detail with the distance to
the camera.
Current maximum shadow precision for local light is about 1 pixel per 0.01
degrees.
For sun light, the maximum resolution is based on the camera far clip
distance which sets the most coarse clipmap.
## Limitation:
Alpha Blended surfaces might not get correct shadowing in some corner
casses. This is to be fixed in another commit.
While resolution is greatly increase, it is still finite. It is virtually
equivalent to one 8K shadow per shadow cube face and per clipmap level.
There is no filtering present for now.
## Parameters:
Shadow Pool Size: In bytes, amount of GPU memory to dedicate to the
shadow pool (is allocated per viewport).
Shadow Scaling: Scale the shadow resolution. Base resolution should
target subpixel accuracy (within the limitation of the technique).
Related to #93220
Related to #104472
