Files
test2/build_files/build_environment/cmake/download.cmake

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

182 lines
5.6 KiB
CMake
Raw Normal View History

# SPDX-FileCopyrightText: 2021-2023 Blender Authors
#
# SPDX-License-Identifier: GPL-2.0-or-later
## Update and uncomment this in the release branch
set(BLENDER_VERSION 4.2)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
function(download_source dep)
set(TARGET_FILE ${${dep}_FILE})
set(TARGET_HASH_TYPE ${${dep}_HASH_TYPE})
set(TARGET_HASH ${${dep}_HASH})
if(PACKAGE_USE_UPSTREAM_SOURCES)
set(TARGET_URI ${${dep}_URI})
elseif(BLENDER_VERSION)
set(TARGET_URI https://projects.blender.org/blender/lib-source/media/branch/blender-v${BLENDER_VERSION}-release/${TARGET_FILE})
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
else()
set(TARGET_URI https://projects.blender.org/blender/lib-source/media/branch/main/${TARGET_FILE})
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
endif()
# Validate all required variables are set and give an explicit error message
# rather than CMake erroring out later on with a more ambigious error.
2022-12-17 13:30:07 +11:00
if(NOT DEFINED TARGET_FILE)
message(FATAL_ERROR "${dep}_FILE variable not set")
endif()
2022-12-17 13:30:07 +11:00
if(NOT DEFINED TARGET_HASH)
message(FATAL_ERROR "${dep}_HASH variable not set")
endif()
2022-12-17 13:30:07 +11:00
if(NOT DEFINED TARGET_HASH_TYPE)
message(FATAL_ERROR "${dep}_HASH_TYPE variable not set")
endif()
2022-12-17 13:30:07 +11:00
if(NOT DEFINED TARGET_URI)
message(FATAL_ERROR "${dep}_URI variable not set")
endif()
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
set(TARGET_FILE ${PACKAGE_DIR}/${TARGET_FILE})
message("Checking source : ${dep} (${TARGET_FILE})")
if(NOT EXISTS ${TARGET_FILE})
message("Checking source : ${dep} - source not found downloading from ${TARGET_URI}")
2022-12-14 14:21:51 +11:00
file(
DOWNLOAD ${TARGET_URI} ${TARGET_FILE}
TIMEOUT 1800 # seconds
EXPECTED_HASH ${TARGET_HASH_TYPE}=${TARGET_HASH}
TLS_VERIFY ON
SHOW_PROGRESS
)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
endif()
if(EXISTS ${TARGET_FILE})
# Sometimes the download fails, but that is not a
# fail condition for "file(DOWNLOAD" it will warn about
2022-12-17 13:30:07 +11:00
# a CRC mismatch and just carry on, we need to explicitly
# catch this and remove the bogus 0 byte file so we can
# retry without having to go find the file and manually
# delete it.
2022-12-17 13:30:07 +11:00
file(SIZE ${TARGET_FILE} TARGET_SIZE)
if(${TARGET_SIZE} EQUAL 0)
file(REMOVE ${TARGET_FILE})
message(FATAL_ERROR "for ${TARGET_FILE} file size 0, download likely failed, deleted...")
endif()
# If we are using sources from the blender repo also
# validate that the hashes match, this takes a
# little more time, but protects us when we are
# building a release package and one of the packages
# is missing or incorrect.
#
# For regular platform maintenaince this is not needed
# since the actual build of the dep will notify the
# platform maintainer if there is a problem with the
# source package and refuse to build.
if(NOT PACKAGE_USE_UPSTREAM_SOURCES OR FORCE_CHECK_HASH)
file(${TARGET_HASH_TYPE} ${TARGET_FILE} LOCAL_HASH)
if(NOT ${TARGET_HASH} STREQUAL ${LOCAL_HASH})
message(FATAL_ERROR "${TARGET_FILE} ${TARGET_HASH_TYPE} mismatch\nExpected\t: ${TARGET_HASH}\nActual\t: ${LOCAL_HASH}")
endif()
endif()
endif()
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
endfunction(download_source)
download_source(ZLIB)
download_source(OPENAL)
download_source(PNG)
download_source(JPEG)
download_source(BOOST)
download_source(BLOSC)
download_source(PTHREADS)
download_source(OPENEXR)
download_source(FREETYPE)
download_source(EPOXY)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
download_source(ALEMBIC)
download_source(OPENSUBDIV)
download_source(SDL)
download_source(OPENCOLLADA)
download_source(OPENCOLORIO)
download_source(MINIZIPNG)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
download_source(LLVM)
download_source(OPENMP)
download_source(OPENIMAGEIO)
download_source(TIFF)
download_source(OSL)
download_source(PYTHON)
download_source(TBB)
download_source(OPENVDB)
download_source(NUMPY)
download_source(LAME)
download_source(OGG)
download_source(VORBIS)
download_source(THEORA)
download_source(FLAC)
download_source(VPX)
download_source(OPUS)
download_source(X264)
download_source(OPENJPEG)
download_source(FFMPEG)
download_source(FFTW)
download_source(ICONV)
download_source(SNDFILE)
download_source(WEBP)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
download_source(SPNAV)
download_source(JEMALLOC)
download_source(XML2)
download_source(YAMLCPP)
download_source(EXPAT)
download_source(PUGIXML)
download_source(FLEXBISON)
download_source(BZIP2)
download_source(FFI)
download_source(LZMA)
download_source(SSL)
download_source(SQLITE)
download_source(EMBREE)
download_source(USD)
download_source(MATERIALX)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
download_source(OIDN)
download_source(LIBGLU)
download_source(MESA)
download_source(NASM)
download_source(XR_OPENXR_SDK)
download_source(WL_PROTOCOLS)
download_source(WAYLAND)
download_source(WAYLAND_LIBDECOR)
download_source(WAYLAND_WESTON)
Fix T86124: Self-hosting external libraries packages Allow downloading of source packages of Blender's dependencies, so that it's easier to provide a "full source archive" that contains the blender source + all dependencies archives. A `make` command for this will be introduced soon. This changes the deps builder slightly to be more flexible with the origin of our source packages. To support this a new CMake variable has been added called `PACKAGE_DIR` where all sources archives will be stored. default: a directory called `packages` in the build folder. alternative-default: if a directory called `packages` exists in the blender source folder that will be used. This is to support the "full source archive" use case. The download phase have been moved from the build phase to the configure phase. Configure will download all sources validate the hashes while downloading. All `[depname].cmake` files have been changed to take a local `file://[path_to_local_tarball]` path rather than a remote URI. A second requirement was that there needed to be an option to grab the sources from the blender SVN mirror rather than upstream. For this an option has been added PACKAGE_USE_UPSTREAM_SOURCES (default ON). The exact location in SVN still needs to be worked out, I tested with my local webserver and codewise it checks out. The path that is in there currently will not work (given there is no mirror there yet). To build this mirror our local package caches can be used. Reviewed By: lazydodo Differential Revision: https://developer.blender.org/D10598
2021-03-16 14:10:30 +01:00
download_source(ISPC)
download_source(GMP)
download_source(POTRACE)
download_source(HARU)
download_source(ZSTD)
download_source(SSE2NEON)
download_source(FLEX)
download_source(BROTLI)
download_source(FMT)
download_source(ROBINMAP)
download_source(IMATH)
download_source(PYSTRING)
download_source(OPENPGL)
download_source(LEVEL_ZERO)
2022-06-29 12:58:04 +02:00
download_source(DPCPP)
download_source(VCINTRINSICS)
download_source(OPENCLHEADERS)
download_source(ICDLOADER)
download_source(MP11)
download_source(SPIRV_HEADERS)
download_source(UNIFIED_RUNTIME)
download_source(UNIFIED_MEMORY_FRAMEWORK)
2022-06-29 12:58:04 +02:00
download_source(IGC)
download_source(IGC_LLVM)
download_source(IGC_OPENCL_CLANG)
download_source(IGC_VCINTRINSICS)
download_source(IGC_SPIRV_HEADERS)
download_source(IGC_SPIRV_TOOLS)
download_source(IGC_SPIRV_TRANSLATOR)
download_source(GMMLIB)
download_source(OCLOC)
download_source(AOM)
download_source(FRIBIDI)
download_source(HARFBUZZ)
download_source(SHADERC)
download_source(SHADERC_SPIRV_TOOLS)
download_source(SHADERC_SPIRV_HEADERS)
download_source(SHADERC_GLSLANG)
download_source(VULKAN_HEADERS)
download_source(VULKAN_LOADER)
download_source(PYBIND11)
download_source(DEFLATE)