Previously, in case of a failure during BVH transfer, when running out
of memory for example, we could get an error such as "BVH failed to
migrate to the GPU due to Embree library error (no error)", because
embree error status was actually reset before being queried.
This commit fixes its propagation.
Pull Request: https://projects.blender.org/blender/blender/pulls/129022
Turns out it is possible to have code to pick up wrong class
when defining a friend:
```
intern\cycles\device/memory.h(255): warning C4099: 'GPUDevice': type name first seen using 'struct' now seen using 'class'
source\blender\gpu\GPU_platform.hh(69): note: see declaration of 'GPUDevice'
```
Now made it so the classes have forward declaration in the CCL
namespace, avoiding possible conflict with the classes with the
same name in the global namespace.
Pull Request: https://projects.blender.org/blender/blender/pulls/128485
Happens for renders from command line, when kernel specialization
thread is still working after the allocators on the Blender side
have been deinitialized.
Add an explicit deinitializaiton, which ensures all Cycles worker
and cache threads are finished before the allocators are deinitialized.
This should solve occasional crashes when running regression tests
for Metal or Metal-RT.
Pull Request: https://projects.blender.org/blender/blender/pulls/128239
This change switches Cycles to an opensource HIP-RT library which
implements hardware ray-tracing. This library is now used on
both Windows and Linux. While there should be no noticeable changes
on Windows, on Linux this adds support for hardware ray-tracing on
AMD GPUs.
The majority of the change is typical platform code to add new
library to the dependency builder, and a change in the way how
ahead-of-time (AoT) kernels are compiled. There are changes in
Cycles itself, but they are rather straightforward: some APIs
changed in the opensource version of the library.
There are a couple of extra files which are needed for this to
work: hiprt02003_6.1_amd.hipfb and oro_compiled_kernels.hipfb.
There are some assumptions in the HIP-RT library about how they
are available. Currently they follow the same rule as AoT
kernels for oneAPI:
- On Windows they are next to blender.exe
- On Linux they are in the lib/ folder
Performance comparison on Ubuntu 22.04.5:
```
GPU: AMD Radeon PRO W7800
Driver: amdgpu-install_6.1.60103-1_all.deb
main hip-rt
attic 0.1414s 0.0932s
barbershop_interior 0.1563s 0.1258s
bistro 0.2134s 0.1597s
bmw27 0.0119s 0.0099s
classroom 0.1006s 0.0803s
fishy_cat 0.0248s 0.0178s
junkshop 0.0916s 0.0713s
koro 0.0589s 0.0720s
monster 0.0435s 0.0385s
pabellon 0.0543s 0.0391s
sponza 0.0223s 0.0180s
spring 0.1026s 1.5145s
victor 0.1901s 0.1239s
wdas_cloud 0.1153s 0.1125s
```
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Co-authored-by: Ray Molenkamp <github@lazydodo.com>
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/121050
Deforming motion blurred point clouds do not render in Cycles
HIP-RT when BVH timesteps != 0 if Blender is launched with
debug memory.
The root cause is that the size of allocated memory for the
bounding boxes is reported to HIP-RT not the number of valid
bounding boxes.
Pull Request: https://projects.blender.org/blender/blender/pulls/127432
IGC 1.0.17384, ocloc 24.31.30508, which:
- add support for Battlemage and Lunar Lake GPUs
- recover from recent performance regression on Linux
- allow to drop older work-around
(9d5164d472) and need for a patched
version on Windows
- ocloc now needs "dg2,mtl" naming for fat binaries.
opencl-clang patches don't get applied anymore by igc build scripts
when llvm is not a git repository, hence I could also drop we can drop
current patch disabling patching.
I've only slightly pushed min-driver-version updates after carefull
testing, instead of jumping to the same version as ocloc as we use to.
Pull Request: https://projects.blender.org/blender/blender/pulls/127251
This new version of the graphics compiler solves a performance
regression on Arc, adds support for Battlemage and Lunar Lake GPUs, and
allows to drop older patch to build fat binaries with broad
compatibility.
This latter change requires using -device dg2,mtl naming instead of
passing architecture ids.
Pull Request: https://projects.blender.org/blender/blender/pulls/127371
The device code was disabled for primitives with deformation blur
and the intersection function always returned false, hence no
rendered primitive.
Other than that, there were a few bugs on both device and host codes
(e.g., the order of current and previous times and the primitive name.)
Pull Request: https://projects.blender.org/blender/blender/pulls/127163
Fixes a few issues with point clouds with HIPRT.
1. Crashing when building the BLAS due to an incorrect sized array.
2. A typo leading to all point cloud intersections being skipped.
3. A typo leading to some motion blurred point clouds rendering
as if they were stationary, or not rendering at all.
Pointclouds, with deformable motion blur, with BVH time steps set to >0
still do not render. Curves seem to have the same issue.
Ref #125086
Pull Request: https://projects.blender.org/blender/blender/pulls/125834
Some of the device memory objects had their host_pointer overwritten
with another CPU-side buffer after allocation. This leads to a leak of
host memory allocated by the device_memory.
There are few remaining places where the host_pointer is assigned and
those seems to be fine because the memory was not yet allocated with
a alloc() call.
While the approach in this change is not very ideal, it is small and
potentially could be ported to the LTS tracks. More ideal solution
would be to utilize device_vector::give_data().
Pull Request: https://projects.blender.org/blender/blender/pulls/126788
OptiX has accepted Catmull-Rom curve data natively since OptiX 7.4, but due to the previous conversion to B-Spline code, the format that data is fed to OptiX wasn't optimal.
Each curve segment was put in the vertex buffer as four independent control points, even though continuous segments actually share control points between each other. This patch compacts that so shared control points only occur once in the vertex buffer.
This compact form uses less memory and also allows OptiX to easily identify segments that belong together into a curve (those where the step between indices is one).
Pull Request: https://projects.blender.org/blender/blender/pulls/125899
The GPU packed state is a static check from the Cycles core perspective,
and it is disabled for non-Apple Silicon GPUs. However, the Metal kernel
always used packed integrator.
This change makes it so the Host and Device side checks for the Host CPU
are aligned, and that Device-side packed state check does not differ from
the Host side.
Pull Request: https://projects.blender.org/blender/blender/pulls/126082
Fixes a crash that can occur if motion blur was on, there is a
deforming mesh in the scene with deformable motion blur turned on,
with BVH time steps set >0.
Render results in my test scene appear to match CPU Embree.
Pull Request: https://projects.blender.org/blender/blender/pulls/125854
This PR fixes live viewport stability issues on Mac when MetalRT is enabled.
There were two sources of instability:
1) `MTLAccelerationStructure` instances were not being correctly retained meaning that use-after-free crashes could occur following a geometry sync.
2) `MTLIntersectionFunctionTable` objects could be unsafely shared between multiple `MetalDeviceQueue` instances (in this case, `setBuffer` being the unsafe mutation)
The solution to 2 involves creating a new `MetalDispatchPipeline` type which is strictly used by only 1 `MetalDeviceQueue` instance.
Pull Request: https://projects.blender.org/blender/blender/pulls/124055
This PR fixes a buffer overrun crash in the MetalRT backend. When non-traceable objects are in the scene, 'num_motion_transforms' is undercounting and the downstream buffer writes (i.e.`motion_transforms[motion_transform_index++]`) are overrunning.
Pull Request: https://projects.blender.org/blender/blender/pulls/124351
Embree device pointer can end up being nullptr even when Embree on GPU is
expected to be used. Previous implementation overlooked this possibility,
leading to a completely silent fallback to the non-Hardware ray-tracing path,
this commit fixes it. We've noticed this as now Embree relies on a driver
component: https://github.com/intel/level-zero-raytracing-support that can
potentially be missing from a system.
Pull Request: https://projects.blender.org/blender/blender/pulls/124085
SYCL runtime currently relies on an internal driver behavior that will
break the driver version string returned by SYCL if it changes:
https://github.com/oneapi-src/unified-runtime/issues/1777
This will be fixed at SYCL runtime level but until we use a new enough
one, we need to add additional verifications to avoid blocking execution
on a driver that will change this internal behavior.
Pull Request: https://projects.blender.org/blender/blender/pulls/124084
This is because with the addition of new features to Cycles, these GPUs
experienced significant performance regressions and bugs, all stemming
from bugs in the Metal GPU driver/compiler. The only reasonable way to
work around these issues was to disable parts of Cycles code on
these GPUs to avoid the driver/compiler bugs.
This resulted in increased development time maintaining these platforms
while being unable to deliver feature parity with other
GPU backends.
It has been decided that this development time is better spent
maintaining platforms that are still actively maintained by
hardware/software vendors, and so AMD and Intel GPU support will be
removed from the Metal backend for Cycles.
Pull Request: https://projects.blender.org/blender/blender/pulls/123551
On some Macs, MNEE would be disabled in Cycles to work around a bug.
However this just led to these devices skipping over MNEE related
parts of the rendering pipeline and not properly progressing through
the render.
This commit fixes this issue by properly disabling MNEE on these devices.
Pull Request: https://projects.blender.org/blender/blender/pulls/123765
Precompiled Cycles kernels make up a considerable fraction of the total size of
Blender builds nowadays. As we add more features and support for more
architectures, this will only continue to increase.
However, since these kernels tend to be quite compressible, we can save a lot
of storage by storing them in compressed form and decompressing the required
kernel(s) during loading.
By using Zstandard compression with a high level, we can get decent compression
ratios (~5x for the current kernels) while keeping decompression time low
(about 30ms in the worse case in my tests). And since we already require zstd
for Blender, this doesn't introduce a new dependency.
While the main improvement is to the size of the extracted Blender installation
(which is reduced by ~400-500MB currently), this also shrinks the download on
Windows, since .zip's deflate compression is less effective. It doesn't help on
Linux since we're already using .tar.xz there, but the smaller installed size
is still a good thing.
See #123522 for initial discussion.
Pull Request: https://projects.blender.org/blender/blender/pulls/123557
The callables generated by OSL reference other external functions
(defined in the OSL services module), in which case OptiX cannot
calculate the right stack size just based on the callable alone, it needs to
know all functions linked together in the pipeline to get to an accurate
result. `optixProgramGroupGetStackSize` has an optional pipeline
argument for this purpose, so make use of that to ensure the correct
stack size is calculated.
Ref #122779
Pull Request: https://projects.blender.org/blender/blender/pulls/123368
Since #118841 there are more cases where Cycles would check for the
graphics interop support. This could lead to a crash when graphics
interop functions are called without having active graphics context.
This change makes it so there is no graphics interop calls when doing
headless render. In order to achieve this the device creation is now
aware of the headless mode.
Pull Request: https://projects.blender.org/blender/blender/pulls/122844
A regression since #118841.
It is possible that the selected preference device is not found, in which
case a default-initialized DeviceInfo would have added to the list. This
device is set to CPU, but with differnet other fields (such as description)
compared to the actual CPU device.
Pull Request: https://projects.blender.org/blender/blender/pulls/122701
ZES_ENABLE_SYSMAN is supposed to be set for free_memory queries to be
available.
These queries are then optionally used since
759bb6c768, for the host memory fallback
feature.
Setting SYCL_ENABLE_PCI was leading ZES_ENABLE_SYSMAN to be set by DPCPP
2022-12 but it's not used by newer versions of DPCPP.
We however temporarily disable SYSMAN by default on Linux as builds with
JEMALLOC enabled currently lead to driver runtime issues. These can be
worked around by using LD_PRELOAD=libigsc.so.
This enables scenes with all textures not fitting in GPU
memory to finally render. For scenes that are fitting,
no functional change or performance change is expected.
Pull Request: https://projects.blender.org/blender/blender/pulls/122385
This patch adds a "shadow" prefix & array index suffixes to the shadow integrator state buffer names. This eliminates confusion when looking at GPU traces etc.
Pull Request: https://projects.blender.org/blender/blender/pulls/121745
This PR contains optimisations and a general tidy-up of the MetalRT backend.
- Currently `scene_intersect` is used for both normal and (opaque) shadow rays, however the usage patterns are different enough to warrant specialisation. Shadow intersection tests (flagged with `PATH_RAY_SHADOW_OPAQUE`) only need a bool result, but need a larger "self" payload in order to exclude hits against target lights. By specialising we can minimise the payload size in each case (which is helps performance) and avoid some dynamic branching. This PR introduces a new `scene_intersect_shadow` function which is specialised in Metal, and currently redirects to `scene_intersect` in the other backends.
- Currently `scene_intersect_local` is implemented for worst-case payload requirements as demanded by `subsurface_disk` (where `max_hits` is 4). The random_walk case only demands 1 hit result which we can retrieve directly from the intersector object (rather than stashing it in the payload). By specialising, we significantly reduce the payload size for random_walk queries, which has a big impact on performance. Additionally, we only need to use a custom intersection function for the first ray test in a random walk (for self-primitive filtering), so this PR forces faster `opaque` intersection testing for all but the first random walk test.
- Currently `scene_intersect_volume` has a lot of redundant code to handle non-triangle primitives despite volumes only being enclosed by trimeshes. This PR removes this code.
Additionally, this PR tidies up the convoluted intersection function linking code, removes some redundant intersection handlers, and uses more consistent naming of intersection functions.
On a M3 MacBook Pro, these changes give 2-3% performance increase on typical scenes with opaque trimesh materials (e.g. barbershop, classroom junkshop), but can give over 15% performance increase for certain scenes using random walk SSS (e.g. monster).
Pull Request: https://projects.blender.org/blender/blender/pulls/121397
This enables the new lazy module loading behavior introduced in OIDN 2.3,
without breaking compatibility with older versions of OIDN (using separate
code paths).
Also, the detection of OIDN support for devices is now much cleaner, and
devices do not need to be matched by PCI address or device name anymore.
Pull Request: https://projects.blender.org/blender/blender/pulls/121362
This PR replaces the existing CPU wall-clock based profiling mechanism with more precise GPU counter based timestamps. As before, it is enabled by setting the env var `CYCLES_METAL_PROFILING=1`. Original implementation by Morteza Mostajabodaveh.
Pull Request: https://projects.blender.org/blender/blender/pulls/121208
get_apple_gpu_architecture will now report if the GPU being checked
is not an Apple GPU.
At the moment this has no functional changes. But it reduces the
chances of mistakes in the future where a developer tries to enable
a feature on newer Apple GPUs using get_apple_gpu_architecture,
and accidentally enables it on unsupported AMD and Intel GPUs.
Pull Request: https://projects.blender.org/blender/blender/pulls/120448
With the switch to using the primary CUDA context it became possible
for peer access between CUDA devices to already have been enabled for
that context, either by a previous Cycles session or third-party library,
thus causing the call to `cuCtxEnablePeerAccess` to return
`CUDA_ERROR_PEER_ACCESS_ALREADY_ENABLED`. This is not a failure
state however, so just needs to be handled like a success return value.
Pull Request: https://projects.blender.org/blender/blender/pulls/120255
Ever since commit [1], `use_metalrt_by_default` will be True
if the GPU being used is not a M1 or M2 based system.
The intention of this was to enable MetalRT by default for
M3 and newer devices that have hardware for ray traversal.
However the side effect of this change was that all AMD GPUs would
have `use_metalrt_by_default` set to True. Which appears to be the
main culprit causing crashes on older AMD GPUs in #120126.
Since these GPUs don't support MetalRT.
This commit fixes this issue by only setting
`use_metalrt_by_default` to True if the GPU is not M1 or M2 based,
and the GPU is Apple Silicon based. Which equates to M3 or newer.
Which is the original intent of this code.
This resolves the issue where AMD GPUs were being told to use MetalRT
by default, when they shouldn't be.
[1] 322a2f7b12
Pull Request: https://projects.blender.org/blender/blender/pulls/120299
