This upgrade for DPC++ and its dependencies brings several
bug fixes and also eliminates some patches and backports
that are now obsolete with this new release.
Pull Request: https://projects.blender.org/blender/blender/pulls/138173
on a freshly installed environment diff.exe requires newer runtime
dll's. For now only fix this on X64 as on windows on arm there's
issues with newer msys2 versions (but I do now know exactly what
they are anthony has brought it up in the past) this can however
not remain in a broken state, so fixing it for X64 only.
Also fix typo in download_package function
These two fixes together should bring a fresh vmprep install back
to working order.
While shaderc is a c++ library, its api is a C based one so
we do not need a debug version of this library, the blender
side was already updated in f2b9dd3462 this cleans up the
builder side of things.
This new version brings several fixes that Blender no
longer needs to patch manually. In addition, it includes
an internal change related to GPU memory management which,
together with the new API, could address several tickets
and issues currently reported in relation to Embree GPU execution.
Pull Request: https://projects.blender.org/blender/blender/pulls/138176
Split out from #138161.
I checked with a locally built OSL (on Linux), and all tests (incl. OptiX OSL)
still pass without the Cycles-side changes in that PR, so we can merge this and
update the libs separately.
The only file that needs to be updated in the deps is `liboslexec.so`,
and probably `llvm/lib/clang/17/include/__clang_cuda_device_functions.h`
(we don't use this when building Blender, but since it's changed,
it's probably cleaner to update it anyways).
Pull Request: https://projects.blender.org/blender/blender/pulls/138788
Broken by PR#134221
- Couple of missing deps
- Windows is not happy with -J when building python deps
as multiple processes will try to write to the same .pdb
- Wrong .lib file name for the zstandard python module.
There are two parts for this PR. One is to change some of our build pipeline to make certain libs reproducible. For this part I want to clarify two things:
1. Why change python to use `--disable-optimizations`?
This is because `--enable-optimizations` turns on PGO (Profile Guided Optimization). PGO is sadly not deterministic and will create different binaries on every recompile. So to create reproducible build this needs to be turned off. This also seems to only have been turned on for Linux specifically(?) on our side. So on Windows and Mac our python build already doesn't have PGO.
2. Why split out cython and zstandard from site-packages?
Sadly pip does not seem to respect `SOURCE_DATE_EPOCH`. It also creates temporary folders with random hashes in them that is then recorded into the Cython libraries (I'll touch on this again later). I've looked at the discussions about this upstream and sadly the pip maintainers do not really want people to use pip as a reproducible build system pipeline and instead directs users to other solutions if they want reproducible builds.
The other part is about setting up our pipeline to not introduce any random hashes or build timestamps into our libraries. Here I do two things:
1. We need to set the `SOURCE_DATE_EPOCH` environmental variable to a specific date that will not change.
This is needed as the compile time date is recorded in certain libraries and files. (So hard coding it with this env var will make the end result reproducible)
2. We need to strip the created static and shared libraries. This is because the static libraries are not created in a deterministic way. For shared libraries some of our libraries includes debug symbols which contains paths to temporary files with random hashes. To solve this without stripping in post, we would need to either patch the linker on Rocky8 or patch a lot of our libraries. I think it is better to just do this as a post build step. (This seems to be what most linux distributions do as well).
With all this, we can make our Linux library builds is almost 100% reproducible. (At least on my machine where I tested)
By almost, I mean that there is sadly a catch in that certain libraries like Cython saves the source code path in their libraries for error messages. However now the builds are reproducible if the folder path is the same.
IE if the libraries are always built in `/home/builder/build_linux/deps_x64`, then they should now be reproducible.
Pull Request: https://projects.blender.org/blender/blender/pulls/134221
This patch replaces our own FindTBB module with TBB's own TBBConfig
module. On the technical side of things, this harvests the TBB CMake
config modules, and switches our TBB CMake find_package calls from
Module mode to Config mode. Integration was done using OpenPGL as a
blueprint for parsing TBB target's property back into CMake variables
(TBB_LIBRARIES and TBB_INCLUDE_DIRS).
Pull Request: https://projects.blender.org/blender/blender/pulls/137063
This PR adds a simple patch to fix libffi compilation errors on Xcode
16.3 / Apple Clang 17. This patch has been ported from libffi Github
PR #857[1] and has now since been included in libffi 3.4.7
[1]: https://github.com/libffi/libffi/pull/857
The reason why we patch libffi v3.4.4 instead of just upgrading to 3.4.7
is that Python require specific versions of its dependencies. (See
comment in build_environment/cmake/versions.cmake about updating Python
version)
Once we upgrade Python and libffi is updated to 3.4.7 or beyond,
this patch can be safely removed.
Pull Request: https://projects.blender.org/blender/blender/pulls/136934
It's better for performance to use a single thread pool for all areas of
Blender, and this gets us closer to that.
Bullet, Quadriflow, Mantaflow and Ceres still contain OpenMP code, but it
was already disabled.
On macOS, our OpenMP libraries are no longer compatible with the latest
Xcode 16.3. By removing OpenMP we no longer have to solve that problem.
OpenMP was disabled for bpy module builds on Windows ARM64, which also no
longer needs to be solved.
Pull Request: https://projects.blender.org/blender/blender/pulls/136865
bpy is not compatible with numpy 2+, as the VFX reference platform uses
1.26 atm. This fix amended the install requirement package to specify
numpy>=1.26,<2.0 to mitigate this issue.
Suggested by Brandy Johnston (@bradyajohnston)
Pull Request: https://projects.blender.org/blender/blender/pulls/134551
PR as discussed on the thread for #134676.
Works by disabling dependent load flag in OIDN, and also switching off OpenMP.
The OIDN bit is just a workaround until they can resolve this properly on their side by fixing the manifest.
The OpenMP bit requires more investigation regarding delayed loading.
Pull Request: https://projects.blender.org/blender/blender/pulls/135319
Quite obscure issue, seemingly caused by the fact that HIP-RT is
passing a large (about 20 MB) global array to a different library
(HIP driver, via hipModuleLoadData). Having global variables of
such size seems to be always problematic as they can not be stored
on stack and, possibly, extra mapping is involved here. It is not
clear whether it is a quirk of the HIP driver, or Linux, or, maybe,
something completely different.
It is possible to work-around the problem by making a temporary
copy of data on heap memory and pass it to the hipModuleLoadData().
This is how other areas are dealing with modules in Blender.
This change contains patch against HIP-RT and the new HIP-RT
library compiled with the patch. It seems to fix the problem
reported in the report.
This change does not resolve OIDN on HIP GPU which seems to have
the same issue. However, it is not a recent regression and the
bug with OIDN GPU denoising can be reproduced using Blender 4.3.
Pull Request: https://projects.blender.org/blender/blender/pulls/135403
This change brings the following improvements on the user level
- Support of GPUs with gfx12 architecture
- New HIP-RT library which in addition to the gfx12 support brings
various bug-fixes.
The known limitation of gfx12 is that OpenImageDenoiser does not yet
support this GPU architecture. This means that while Cycles will use the
full advantage of the gfx12 (including hardware accelerated ray-tracing),
denoising will only be possible on CPU, or secondary gfx11 or below GPU.
This is something that requires a change in OIDN and it is to late to do
it for Blender 4.4, but it is something to look forward for Blender 4.5.
The gfx12 changes for the pre-compiled kernels is rather trivial,
so it comes together (in the same PR) as the bigger HIP-RT change.
On the development side this change brings the following improvements:
- One step compile and link (much simpler CMake rules)
- Embedding BVH binaries in hiprt dll (which makes it easier to package
and load, without relying on special path configuration)
Co-authored-by: Sahar Kashi <sahar.kashi@amd.com>
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/133129
the static crt was selected, which isn't a problem as crt
object aren't passed between the host and the shared lib
still it's be nice to use the same runtime between both.
`SHADERC_ENABLE_SHARED_CRT` is a msvc specific option and
doesn't affect other platforms.
also switch build system from msbuild to ninja on windows.
this brings down the shaderc build time from 185 to 47 seconds
for the release libs, and 224 to 53 for the debug libs
total savings: 5 minutes!
OpenCollada used to rely on PCRE (a Perl Regexp library). Since
switching to Aras' OpenCollada fork (#122270), the library is no
longer needed, but is still required as a dependency.
This patch cleans this up by completely removing it from our build
system and linux system package installation script. This also lets
us remove it from our pre-compiled library platform repos, making the
process of recompiling our libraries from scratch easier as it wasn't
compiled by our dependency builder anymore.
Pull Request: https://projects.blender.org/blender/blender/pulls/134310
