Commit Graph

3158 Commits

Author SHA1 Message Date
Ray Molenkamp
14c218b3fd deps_builder: fix hardcoded path
the windows builder has 2 different build folders for the debug and
release libs, hardcoding the path like this leads to an extra 1GB
of downloaded files. The windows builder supplies a single PACKAGE_DIR
for both but gets ignored.

This change promotes PACKAGE_DIR to be cached in CMakeCache so it can
be changed if needed.

Differential Revision: https://developer.blender.org/D16213

Reviewed by; brecht, dfelinto
2022-10-25 17:00:48 -06:00
Campbell Barton
3e292404d2 Build: fix/workaround for the opencollada.diff not applying on Linux 2022-10-25 17:51:30 +11:00
Ray Molenkamp
04eab0fd01 deps_builder: expose FORCE_CHECK_HASH as an option
The check was already there, but given it was not
exposed as an option it didn't show up in my cmakecache
and i got tired of having to lookup the exact name
all the time.
2022-10-21 12:48:17 -06:00
Brecht Van Lommel
e7a6917617 Build: add option to error when features can't be enabled
This is to help ensure buildbot builds are correct, while still gracefully
disabling features in user/developer builds.

* Add WITH_STRICT_BUILD_OPTIONS to give an error when features can't be
  enabled due to missing libraries or other reasons. Add new macro
  set_and_warn_library_found used everywhere features were being
  automatically disabled.

* Remove code from Windows and macOS for various libraries that would
  automatically disable features. set_and_warn_library_found could be
  used here also, but we are generally assuming the precompiled libraries
  are complete and only test for availability when libraries are just
  added.

Differential Revision: https://developer.blender.org/D16104
2022-10-21 20:04:47 +02:00
Brecht Van Lommel
ebe2319692 Build: disable JACK option in macOS releases
It has not actually been enabled there for a long time in official releases,
so this just fixes the warning. Making it work again would be good, but for
now JACK is only supported on Linux.
2022-10-21 20:04:47 +02:00
Sergey Sharybin
56d1f0772d Fix missing OpenMP in Linux builds
OpenSubdiv OpenMP detection was interfering, but we don't use it there
anymore so just remove that code.
2022-10-21 20:04:47 +02:00
Ray Molenkamp
116d7b0042 make.bat: update the libraries before calling update_sources.py
The issue we ran into a lot is we have a python script that updates
git+SVN. Which works fine most of the time, except when we have a
python update in SVN, or worse a python version change. Python really
doesn't enjoy having its files being deleted or changed while it is
running and users generally end up with a corrupted lib folder.

This change updates the library folder using svn.exe first before
letting the python script run sidestepping the issue in most cases.

The python script will still run and do the more elaborate work
like updating git and switching SVN branches which could still
run into issues cause python still doesn't like being changed
while running but there's not a whole lot we can about that,
for *most* people however things will just work now.
2022-10-21 09:50:14 -06:00
Xavier Hallade
0cfac5b043 Cycles: oneAPI: migrate from deprecated APIs, require libSYCL 6.0+
sycl::info::device::ext_intel_* descriptors are deprecated,
replaced with sycl::ext::intel::info::device:: that are available from
6.0+, for which we now check version in CMake.
2022-10-21 15:36:49 +02:00
Ray Molenkamp
3ccec47841 cmake/windows: Prepare for 3.4 library changes for opencollada
The opencollada dependency will be using an external xml2 library
for 3.4. This change allows to build against both old and new
style lib folders.

As it is a C library it did not need a special debug version.
2022-10-21 07:04:21 -06:00
Sergey Sharybin
4bfb99e4d8 Cycles: Bump versions of DPC++, IGC, and dependencies
Patch by Xavier Hallade. Committing next to the actual libraries update
in the svn.
2022-10-21 12:19:54 +02:00
Bastien Montagne
5814e35af1 install_deps: Update OSL for Blender 2.4 release.
As requested by T101403, OSL version is now 1.12.6.2.
2022-10-21 08:51:23 +02:00
Ray Molenkamp
dc09cc13ea deps_builder: add missing OSLNoise library on windows
OSLNoise is a new library, was missing in the debug
configuration on windows.
2022-10-20 18:31:15 -06:00
Ray Molenkamp
6515177981 deps_builder: OpenCollada fixes for windows
- build and use our version of libxml

- the cli tools had a linker error due
to it trying to link a shared version
of libxml, disabled both and zlib 1.2.3
with a patch since we do not want/need
them for blender.

- postfix the libraries with _d for debug
automatically so we don't have to fix that
during the harvest.

due to this only being windows changes no
rebuild needed for the other platforms.
2022-10-20 18:29:59 -06:00
Ray Molenkamp
8c361d6f66 deps_builder: fix missing png harvest on windows 2022-10-20 11:46:42 -06:00
Bastien Montagne
3bac6f17f1 install_deps: Update Python, OIIO, openPGL and FFMPEG versions.
Update libraries for Blender 3.4, according to T101403:
* python: 3.10.8
* OIIO: 2.3.20.0
* OpenPGL: 0.4.0-beta
* FFMpeg: 5.1.2
2022-10-20 17:07:53 +02:00
Brecht Van Lommel
0d7065504e Build: mark remaining CVEs reported by cve_check as mitigated or ignored
After the last library update cve_check still reported some false positives.
One GMP issues was mitigated with a patch in the library update. The others
are ignored, with a description explaining why they do not affect Blender.

Ref D16269, T101403
2022-10-20 13:57:45 +02:00
Ray Molenkamp
21deba56aa Build: update various libraries for 3.4, fixing bugs and security issues
THis is bumping dependencies to fix known CVEs, with the exception of
OpenImageIO which also includes bugfixes for performance and correctness
with some image types.

zlib 1.2.12 -> 1.2.13
freetype 2.11.1 -> 2.12.1
openimageio 2.3.13.0 -> 2.3.20.0
python 3.10.2 -> 3.10.8
openjpeg 2.4.0 -> 2.5.0
ffmpeg 5.0 -> 5.1.2
sndfile 1.0.28 -> 1.1.0
xml2 2.9.10 -> 2.10.3
expat 2.4.4 -> 2.4.9
openssl 1.1.1g/i -> 1.1.1q
sqlite 3.31.1 -> 3.37.2

Notable changes:
* AOM: the hack we had in place to make it not detect pthreads on windows no
  longer worked with a more recent cmake version. Disabled pthreads with a
  diff on Windows.
* Python: embedded copy of zlib 2.1.12 swapped out for our 2.1.13 copy with
  some folder manipulation on Windows.
* Freetype: was harbouring a copy of zlib 2.1.12 as well, so that had to end.
* FFmpeg: patch used to fix D11796 is no longer needed. Add new patch to deal
  with simple_idct.asm generating an object file with no sections in it,
  backport from upstream commit.
* TinyXML: still being downloaded but no longer used by OpenColorIO, removed.
* GMP applied upstream patch to fix CVE-2021-43618, as there is no release yet.
* SQLite and Libsndfile patches no longer needed.

Includes contributes by Ray Molenkamp, Campbell Barton and Brecht Van Lommel.

Ref T101403

Differential Revision: https://developer.blender.org/D16269
2022-10-20 13:56:01 +02:00
Ray Molenkamp
debacbfe5c Build: update OpenPGL hash for re-tagged version without symbols conflicts
Ref D16269, T101403
2022-10-20 13:56:01 +02:00
Ray Molenkamp
303dd263e2 Build: update OSL to version 1.12.6.2
Required changes to make OSL pick up the new oslnoise library.

Ref D16269, T101403
2022-10-20 13:56:01 +02:00
Xavier Hallade
d763e294fc Cycles: oneAPI: fix libsycl.so files harvesting
was missing libsycl.so.5.7.0-16 when using such version.
2022-10-19 16:42:10 +02:00
Brecht Van Lommel
c2fe387244 Build: get make deps working with Xcode command line tools
Deduplicating code with Xcode detection for Blender builds.
2022-10-18 14:17:28 +02:00
Brecht Van Lommel
3eaf2b7fc6 Fix OpenPGL and OneAPI being detected by CMake when Cycles is disabled 2022-10-17 14:34:25 +02:00
Brecht Van Lommel
baa9a00f5f Fix Wayland not being disabled when dependencies are not found 2022-10-17 14:34:25 +02:00
Sebastian Herholz
a90b0eb772 Deps: Updating OpenPGL 0.4.0 hash value 2022-10-14 21:10:55 +02:00
Campbell Barton
5d53e6b3c1 Deps: include libxkbcommon-devel for Linux build script, remove wayland
- Include libxkbcommon headers: needed for building Blender with Wayland.
- Remove Wayland libraries (now wayland is built locally).
- Split up packages into two lists, one for building libs,
  another for building Blender - since it's useful to be able to build
  Blender, testing the libs work as expected.
2022-10-14 19:12:11 +11:00
Campbell Barton
06081d48e3 Deps: add comments noting why packages are installed
Some don't seem to be needed anymore, noted these in comments too.
2022-10-14 18:59:12 +11:00
Campbell Barton
b2ab9977bf Deps: use the locally built wayland-scanner for wayland_protocols
This removes the dependency on wayland-devel, with some potential
for errors caused from using two different versions.

Note that the generated files are currently unchanged.
2022-10-14 18:59:11 +11:00
Sebastian Herholz
b2627dff49 Build: Upgrading OpenPGL version to 0.4.0 2022-10-13 17:45:14 +02:00
Bastien Montagne
d292d635ea install_deps: Add system libdecor dev package.
ref. T101403
2022-10-13 14:49:29 +02:00
Campbell Barton
375dafe3ef Build: use bash for build environment script
- Prefer bash over sh to so it's unambiguous which features can be used.
- Use array syntax to specify main package list, allowing to comment
  why packages are required and making git-blame easier to track down
  commits that add/remove packages.
- Add license header.
2022-10-13 13:20:58 +11:00
Ray Molenkamp
4fc4005f6b deps_builder: add missing openpgl config file
harvesting openpgl_Exports-debug.cmake was missing for
windows in the build script, and we manually added it
to svn when openpgl landed when we realized it was
missing.

This fixes the issue at the root of the problem, so
the file will not be missing for future updates.
2022-10-11 09:14:10 -06:00
Sebastian Parborg
c425599216 Fix linker order of X11 and Wayland libs
For some compiler and linker configurations, linking would fail as the
wayland libs were linked at a high level and not at the actual code
where they were needed.

After talking to Campbell, we decided to clean up this part and now
only link both the X11 and Wayland libs where they are used.
2022-10-11 14:52:35 +02:00
Ray Molenkamp
9b8be81eef deps_builder: Add support for cve-bin-tool
This change adds support for intels cve-bin-tool [1]
in the deps builder. This adds 2 new targets to the
builder that do not build automatically but can be
build on demand when required.

`make cve_check` will output to the console.
`make cve_check_html` will output a html file that
can be shared with other people.

Requirements:

- A working installation of cve-bin-tool on the system

Not required but higly recommended:

- Obtaining a key from the nvd [2] to speed up the
  database download. you can pass the key to cmake
  using `-DCVE_CHECK_NVD_KEY=your_api_key`

[1] https://github.com/intel/cve-bin-tool
[2] https://nvd.nist.gov/developers/request-an-api-key

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D16160
2022-10-10 11:48:05 -06:00
Campbell Barton
d779792977 Build: reference bundled wayland and libdecor headers
When using pre-compiled libs, reference the bundled wayland headers,
needed so the headers from the bundled wayland-scanner are compatible.

Part of D16091.
2022-10-08 08:00:17 +11:00
Campbell Barton
66f184d981 Deps: add libdecor for Wayland window frames
Include libdecor, note that this only bundles a header file
as Blender dynamically loads on the system at run-time.

Part of D16091.
2022-10-08 08:00:08 +11:00
Campbell Barton
b083755c20 Deps: harvest Wayland headers, build Wayland libraries
The headers on centOS7 (Wayland 1.15) aren't compatible with the headers
generated by wayland-scanner 1.20.

Include the headers to support building Wayland on systems with
older Wayland installed.

Using these headers will be part of a separate commit.

Part of D16091.
2022-10-08 08:00:04 +11:00
Xavier Hallade
3714d3c3ce Cycles: link oneAPI backend with debug version of sycl when in Debug
It fixes SYCL runtime issues in Debug builds that were due to mixing
Release and Debug MSVC runtimes.
This commit also removes specific handling of dpcpp compiler executable
to simplify the CMake implementation. Using it like clang++ works and
clang++ executable is also available from Intel oneAPI DPC++ compiler in
case it doesn't.
2022-10-07 16:14:50 +02:00
Xavier Hallade
7eeeaec6da Cycles: use direct linking for oneAPI backend
This is a minimal set of changes, allowing a lot of cleanup that can
happen afterward as it allows sycl method and objects to be used outside
of kernel.cpp.

Reviewed By: brecht, sergey

Differential Revision: https://developer.blender.org/D15397
2022-10-07 09:50:05 +02:00
Dalai Felinto
bd77cdde58 Build Environment: Update Ubuntu instructions to include bison
Ubuntu also requires bison (just like macOS seems to do).
2022-10-06 18:09:29 +02:00
Sybren A. Stüvel
29e1cd75fa Linux CentOS 7 setup script
Add a script to set up a Linux CentOS 7 machine so that it can be used
to build the precompiled libraries for Linux.

Reviewed By: sergey, brecht

Differential Revision: https://developer.blender.org/D16171
2022-10-06 16:59:07 +02:00
Ray Molenkamp
74ff0aeea0 deps_builder: harden the package download process
During the 3.3 release some packages were missing
in SVN during the release and it ended up building
the release tarball without issues when re-running
the `make source_archive_complete` command after it
failed initially. The tarball however had 0 byte files
for the missing packages.... not good.

This diff hardens the download process by :

1) Validating all required variables are set. This
catches the erroneously attempt at downloading the
nanovdb package even though we have removed it
from versions.cmake but neglected to remove it
from download.cmake

2) When a download fails (due to either a missing
package, or bad download URL) FILE Download will
warn about a hash mismatch but will carry on
happily, you then have to go into the file system
go delete the 0 byte file to retry the download.
We know for a fact the file is bad when it is 0
bytes, just delete it.

3) When we are using the blender repository
(and likely building a source archive) explicitly
validate the hash of all packages. Normally the
build process does this, however when building
a source archive the build does not actually
run for a dep. So preform this check during the
configuration stage.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D16124
2022-10-05 12:05:26 -06:00
Bastien Montagne
195bcea0d0 install_deps: Fix typo in previous commit.
Typo from rB7fa0080c4ffa.
2022-10-05 17:19:16 +02:00
Bastien Montagne
7fa0080c4f Install_deps: Add building of OpenPGL.
Ref. T101403.
2022-10-05 12:08:05 +02:00
Campbell Barton
41281cc0e7 Build: fix missing xml2 build time dependency for wayland
Also prepend the existing 'PKG_CONFIG_PATH' instead of overwriting it
as this prevents 'devtoolset-9' libraries being used in the release
environment.
2022-10-05 08:48:32 +11:00
Brecht Van Lommel
ea95d04245 Build: integrate OpenPGL into platform files like other libraries
To avoid issues with install_deps. If we more generally switch to using
CMake configs then perhaps this code can be deduplicated again or at
least simplified.
2022-10-04 20:24:27 +02:00
Bastien Montagne
4d9588ee45 Revert "install_deps: update OpenVDB for Blender 3.4."
This update has been moved back to Blender 3.5.

This reverts commit ded4604d71.
2022-10-04 17:35:21 +02:00
Bastien Montagne
25d162f864 Revert "install_deps: update OIIO/Boost/Python/NumPy versions."
These updates have been moved back to Blender 3.5.

This reverts commit 1e1e9014cf.
2022-10-04 17:35:21 +02:00
Campbell Barton
0c282c068f Cleanup: cmake indentation 2022-09-29 13:32:16 +10:00
Brecht Van Lommel
e191cf89b2 Fix macOS cmake warning about missing OpenAL, after Python module changes 2022-09-28 14:39:25 +02:00
Sebastian Herhoz
75a6d3abf7 Cycles: add Path Guiding on CPU through Intel OpenPGL
This adds path guiding features into Cycles by integrating Intel's Open Path
Guiding Library. It can be enabled in the Sampling > Path Guiding panel in the
render properties.

This feature helps reduce noise in scenes where finding a path to light is
difficult for regular path tracing.

The current implementation supports guiding directional sampling decisions on
surfaces, when the material contains a least one diffuse component, and in
volumes with isotropic and anisotropic Henyey-Greenstein phase functions.

On surfaces, the guided sampling decision is proportional to the product of
the incident radiance and the normal-oriented cosine lobe and in volumes it
is proportional to the product of the incident radiance and the phase function.

The incident radiance field of a scene is learned and updated during rendering
after each per-frame rendering iteration/progression.

At the moment, path guiding is only supported by the CPU backend. Support for
GPU backends will be added in future versions of OpenPGL.

Ref T92571

Differential Revision: https://developer.blender.org/D15286
2022-09-27 15:56:32 +02:00