Commit Graph

174 Commits

Author SHA1 Message Date
Campbell Barton
b9d4b3b9a4 Cleanup: unused import 2025-05-09 13:36:18 +10:00
Sergey Sharybin
d0bc85487a Cleanup: Formatting of make_update.py 2025-05-06 13:47:41 +02:00
Sergey Sharybin
1ec9aa1cf1 Fix: make update does not update benchmark data
A mistake in the #137219 which cleanup too much code.

Pull Request: https://projects.blender.org/blender/blender/pulls/138483
2025-05-06 12:21:41 +02:00
Sergey Sharybin
617d79be76 Fix: maker test was checking for the old test files location
There is no need to do this anymore, as the files are expected to
always be there.
2025-05-05 16:52:55 +02:00
Brecht Van Lommel
dfccf9510d Build: Auto add Git LFS fallback remote in "make update"
For the official GitHub mirror and other repositories that do not include LFS
files, this adds an `lfs-fallback` remote. It will be used automatically if a
file can't be found on the regular remote.

Ref #137215

Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/137615
2025-05-05 15:33:53 +02:00
Sergey Sharybin
bbfc97ad6f Move tests/data and assets to the main repository
This change moves the tests data files and publish folder of assets
repository to the main blender.git repository as LFS files.

The goal of this change is to eliminate toil of modifying tests,
cherry-picking changes to LFS branches, adding tests as part of a
PR which brings new features or fixes.

More detailed explanation and conversation can be found in the
design task.

Ref #137215

Pull Request: https://projects.blender.org/blender/blender/pulls/137219
2025-05-05 15:10:22 +02:00
Campbell Barton
d616c87d03 Cleanup: spelling in comments (make check_spelling_*) 2025-03-21 11:51:50 +11:00
YimingWu
0566b802c7 Fix #134550: Python: Limit numpy version for bpy wheel
bpy is not compatible with numpy 2+, as the VFX reference platform uses
1.26 atm. This fix amended the install requirement package to specify
numpy>=1.26,<2.0 to mitigate this issue.

Suggested by Brandy Johnston (@bradyajohnston)

Pull Request: https://projects.blender.org/blender/blender/pulls/134551
2025-03-20 15:19:05 +01:00
Sergey Sharybin
44c01ff893 Fix: Incorrect platform tag used for WoA bpy
The platform tag came from virtualized CPU on which Python 3.9 is
running on the buildbot.

Use the same trick to detect the host CPU as for the message on the
splash screen.

Pull Request: https://projects.blender.org/blender/blender/pulls/136085
2025-03-17 16:18:19 +01:00
Campbell Barton
e50420f918 Cleanup: resolve mypy warning 2025-03-06 11:13:09 +11:00
Sebastian Parborg
277add8fc9 Add ability to create a test data archive with make_source_archive
This makes it so that we ship the test data for every major release
in its own separate archive. (In case someone wants to easily run the
tests for a specific older release without using git)

Pull Request: https://projects.blender.org/blender/blender/pulls/135293
2025-03-04 13:55:23 +01:00
Sebastian Parborg
7913237d33 Change make_source_archive to include all submodule files
It will not include submodules that are not checked out by default.
It now also has an explicit variable for folders to skip.

Pull Request: https://projects.blender.org/blender/blender/pulls/135293
2025-03-04 13:55:12 +01:00
Thomas Dinges
d66b10bac0 Cleanup: Remove transform parameter now that asset folder was updated. 2025-02-04 16:43:54 +01:00
Thomas Dinges
21873221f8 Fix make_source_archive using outdated path to assets. 2025-02-04 16:19:54 +01:00
Campbell Barton
4cd827870d Cleanup: quiet check_spelling_* targets
Also correct outdated references to `ghash`.
2025-02-02 13:58:34 +11:00
Campbell Barton
9d4b48b107 Cleanup: spelling in build-files & docs 2025-01-31 15:22:50 +11:00
Brecht Van Lommel
381e125153 Fix: Invalid character in bpy package description on pypi
Unclear why this started happening since the 4.3.0 package, maybe an update
to Python on the build machine.
2025-01-15 09:42:05 +01:00
Campbell Barton
4f1817cc18 Cleanup: declare __all__ for Python scripts
Declare all to make public public API's explicit and
help detect unused code.
2025-01-06 16:45:36 +11:00
Campbell Barton
3091dadfeb Cleanup: declare __all__ for some scripts in build_files/ 2025-01-04 20:37:46 +11:00
Bastien Montagne
2c9ab53273 Add 'system python' validation for some py scripts.
The goal of this test is to try to import some critical py scripts with the
system python of the building machine.
The main target is to ensure that these py scripts remain usable by all
buildbot machines, as some of them are using fairly outdated python
versions.

Current status:
* Scripts in `build_files` and `docs` are checked.
* Some python scripts in `build_files` were 'reverted' to be compatible
  with older required python version currently (3.6).
* A few scripts are excluded from the test, mostly because they use Blender's
  `bpy` module, which means they are only intended to be ran with Blender's
  python anyway.
* The test is only enabled for Linux buildbots currently, as they use the
  oldest Python by far.

Notes:
* Some more scripts are likely to be moved around in the future.
* Whether these tests need to be enabled on windows or macos platforms remains
  an open question.

Pull Request: https://projects.blender.org/blender/blender/pulls/130746
2024-12-24 11:55:29 +01:00
Campbell Barton
6ca1417103 Cleanup: suppress unused Python warnings
Suppress unused warnings using the "vulture" utility.

- Include public definitions in the modules `__all__`.
- Mark arguments & variables as unused with a "_" prefix.
2024-12-03 12:54:13 +11:00
Campbell Barton
bc66905a15 Cleanup: remove/comment unused functions 2024-12-02 11:23:47 +11:00
Campbell Barton
273f48cd53 Cleanup: use main functions to allow importing scripts
Support importing scripts without running their logic to
allow basic validation (see #130746).

Parts of !131037 were used.

Co-authored-by: Bastien Montagne <bastien@blender.org>
2024-11-29 15:54:58 +11:00
Raphael Langerhorst
f662caf722 Add RISCV CPU architecture support to build config
Remove assert statement from make_update.py, making it ready for any
architecture.

Add riscv 32, 64 and 128 bit cpu architecture with little/big endian
to Blender's BLI_build_config.h and Libmv's build_config.h

Tested (to compile) on riscv64 little endian machine.

Pull Request: https://projects.blender.org/blender/blender/pulls/130920
2024-11-28 12:37:26 +01:00
Sergey Sharybin
c533a7b008 Fix: make_bpy_wheel.py is incompatible with Python 3.9
Buildbot uses Python 3.9, so it needs to be supported.

Arguably, the system-wide Python 3.6 needs to be supported to, but
it is a bit more tricky change, and is not a stopper for now.

Pull Request: https://projects.blender.org/blender/blender/pulls/130664
2024-11-21 11:48:43 +01:00
Campbell Barton
3c3c27f1ed Unbreak "make update" on Rocky8 which uses Python 3.6 2024-11-06 21:14:53 +11:00
Campbell Barton
8570dd22e7 Merge branch 'blender-v4.3-release' 2024-11-02 23:06:24 +11:00
Campbell Barton
62a5aed215 Cleanup: quiet mypy type warning 2024-11-02 22:56:57 +11:00
Campbell Barton
a0453ab87a Cleanup: update use of typing in for Python scripts 2024-10-23 12:48:09 +11:00
Campbell Barton
f557b0071f Unbreak "make update" on the buildbot which uses Python 3.9 2024-10-18 13:05:36 +11:00
Campbell Barton
dbfe8db84c Cleanup: remove use of deprecated typing from build_files/ 2024-10-18 11:01:43 +11:00
Brecht Van Lommel
b74dfa8cfc Build: Changes for make deps to work on Linux arm64 again
This is not an officially supported platform, but it was working before
so might as well keep it up to date.

* Tweak logic for various BLENDER_PLATFORM_ARM checks
* Use linux_arm64 name for folders, matching Windows and macOS
* CUDA is enabled, SYCL and HIP are not
* Tested to work on Rocky Linux 8
2024-09-24 15:54:47 +02:00
Sergey Sharybin
0d76071289 Build: Add option to remove stale files from checkout
This change adds `--prune-destructive` command line argument to the
`make_update.py` script which ensures directories which used to be
an in-worktree checkout do not exist. Currently it is addons and
addons_contrib folders.

This gives buildbot an ability to ensure checkout used for build is
correct, regardless of which branch or patch was built there.

The option is needed because sometimes it is still required to be
able to build patches against v330 or v360 which still use the old
style of addons.

Alternative solution could be to ignore the folders in the INSTALL
command, but this could have impact on users who wants to package
custom addons.

The downside of this solution is that buildbot potentially will be
downloading addons and addons_contrib repositories every time it
builds a patch against v330 or v360. It should not be a problem in
practice since the repositories are rather small, and patches for
these tracks are not built very often.

Ref #126557

Pull Request: https://projects.blender.org/blender/blender/pulls/127798
2024-09-18 17:53:25 +02:00
Brecht Van Lommel
97acee6e68 Cleanup: make format 2024-05-27 12:24:16 +02:00
Brecht Van Lommel
412671a765 Fix: Wrong python tag version in bpy pip package
Did not cause known issues, but better to be right.
2024-05-27 11:58:49 +02:00
Brecht Van Lommel
ef5e04c02e Fix #120181: bpy pip package not installing on macOS
Since macOS 11, the minor version should be zero. Previously 10.x were
major releases.
2024-05-27 11:58:49 +02:00
Campbell Barton
c4a0bbb1f4 Extensions: Support online extensions and move add-ons outside Blender
The extensions system allows to extend Blender with connectivity to the internet. Right now it means Blender can
discover and install add-ons and themes directly from the internet, and notify users about their updates.

By default this is disabled (opt-in), and users can enable it the first time they try to install an extension or visit
the Prefences > Extensions tab. If this is enabled, Blender will automatically check for updates for
extensions.blender.org upon startup.

When will Blender access the remote repositories:

* Every time you open the Preferences → Extensions: ALL the enabled repositories get checked for the latest info (json)
* Every time you try to install by dragging: ALL the enabled repositories get checked for the latest info (json).
* Every time you start Blender: selected repositories get checked for the latest info (json).

------------------

From the Blender code point of view, this means that most of the add-ons and themes originally bundled with Blender
will now be available from the online platform, instead of bundled with Blender. The exception are add-ons which are
deemed core functionality which just happened to be written as Python add-ons.

Links:
* Original Extenesions Platform Announcement: https://code.blender.org/2022/10/blender-extensions-platform/
* Extensions website: https://extensions.blender.org/
* User Manual: https://docs.blender.org/manual/en/4.2/extensions/index.html#extensions-index
* Technical specifications: https://developer.blender.org/docs/features/extensions/
* Changes on add-ons bundling: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593

------------------

This PR does the following:

* Move extensions out of experimental.
* No longer install `scripts/addons` & `scripts/addons_contrib`.
* Add `scripts/addons_core` to blender's repository.

These add-ons will still be bundled with Blender and will be always enabled in the future, with their preferences
moved to be more closely integrated with the rest of Blender. This will happen during the remaining bcon2 period.
For more details, see #121830

From scripts/addons:

* copy_global_transform.py
* hydra_storm
* io_anim_bvh
* io_curve_svg
* io_mesh_uv_layout
* io_scene_fbx
* io_scene_gltf2
* pose_library
* ui_translate
* viewport_vr_preview

Extra: bl_pkg (scripts/addons_contrib)

Note: The STL (legacy) add-on is going to be moved to the extensions platform. There is already a C++ version on core
which is enabled by default.

All the other add-ons are already available at extensions.blender.org. To use them you need to:

* Go to User Preferences > Extensions
* You will be greated with an "Online Extensions" message, click on "Enable Repository".
* Search the add-on you are looking for (e.g, Import Images as Planes).
* Click on Install

Over time their maintaince will be transferred over to the community so their development can carry on. If you used to
help maintain a bundled add-on please read: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593

Ref: !121825
2024-05-15 19:26:29 +02:00
Campbell Barton
06adfcaf38 Cleanup: unused imports 2024-04-19 16:09:30 +10:00
Campbell Barton
ae950451ea Cleanup: remove redundant f-string use 2024-04-01 16:51:32 +11:00
Campbell Barton
0eb1317e70 Cleanup: remove unused exception value 2024-03-29 16:18:28 +11:00
Campbell Barton
6a937d646b Cleanup: remove unused variables 2024-03-21 10:11:54 +11:00
Campbell Barton
2e6739967e Cleanup: reduce right-shift in Python scripts
Also place the return type on it's own line as it's easier to identify
when it's in a predictable location instead of the line ending.
2024-02-28 11:02:54 +11:00
Campbell Barton
44e64b8f29 Cleanup: correct type hints for Python scripts 2024-02-28 11:02:53 +11:00
Campbell Barton
4f8db2ee67 Cleanup: use static sets for contains checks, remove f-string use 2024-02-28 11:02:49 +11:00
Sergey Sharybin
05b465f02d Merge branch 'blender-v4.1-release' 2024-02-27 20:02:45 +01:00
Sergey Sharybin
468e4b0e3b Make update: Bring behavior closer to what it used to be
The `make update` will now always update all initialized submodules,
including the library ones.

The `make_update.py --no-libraries` will neither initialize nor update
pre-compiled libraries or tests.

Pull Request: https://projects.blender.org/blender/blender/pulls/118812
2024-02-27 20:02:15 +01:00
Sergey Sharybin
ca63ce835f Merge branch 'blender-v4.1-release' 2024-02-23 17:41:22 +01:00
Sergey Sharybin
9277377f6b Make update: Show Git LFS download progress
Solves the issue of the script potentially sitting for a long time
without having any progress reported. Confusingly, such behavior
depends on Git version.

In older versions (< 2.33) there will be progress reported, but it
then got changed by the commit in Git:

  7a132c628e

The delayed checkout is exactly how Git LFS integrates into the Git
process. Another affecting factor for the behavior is that submodule
configured to use "checkout" update policy is forced to have quite
flag passed to the "git checkout":

  https://github.com/git/git/blob/v2.43.2/builtin/submodule--helper.c#L2258

This is done to avoid the long message at the end of checkout about
the detached state of HEAD, with instructions how to resolve that.

There are two possible solutions: either use "rebase" update policy
for submodules, or skip Git LFS download during the submodule update.

Changing the update policy is possible, but it needs to be done with
a bit of care, and possible revised process for updating/merging
tests data.

This change follows the second idea of delaying LFS download for a
later step, so the process is the following:
- Run `git submodule update`, but tell Git LFS to not resolve the links
  by using GIT_LFS_SKIP_SMUDGE=1 environment variable.
- Run `git lfs pull` for the submodule, to resolve the links.

Doing so bypasses hardcoded silencing in the Git. It also potentially
allows to recover from an aborted download process.

The `git lfs pull` seems to be a nominal step to resolve the LFS links
after the smudging has been skipped. It is also how in earlier Git
versions some Windows limitations were bypassed:

  https://www.mankier.com/7/git-lfs-faq

The submodule update now also receives the "--progress" flag, which
logs the initial Git repository checkout process, which further
improves the feedback.

The byproduct of this change is that an error during precompiled
libraries and tests data update is not considered to be fatal.
It seems to be more fitting with other update steps, and allows
more easily reuse some code.

There is also a cosmetic change: the messages at the end of the
update process now have their own header, allowing more easily
see them in the wall-of-text.

Pull Request: https://projects.blender.org/blender/blender/pulls/118673
2024-02-23 17:40:59 +01:00
Sergey Sharybin
9b2c312ac7 Merge branch 'blender-v4.1-release' 2024-02-23 12:33:11 +01:00
Sergey Sharybin
63613e431e Fix make update pulling libraries when --no-libraries is provided
Likewise, skip tests update when --use-tests is not provided.

It was a bit of ambiguous situation because libraries and tests
are technically submodules. After some feedback it seems that it
is better to ignore submodule for libraries and tests unless
requested explicitly.

Pull Request: https://projects.blender.org/blender/blender/pulls/118631
2024-02-23 12:24:22 +01:00