For the official GitHub mirror and other repositories that do not include LFS
files, this adds an `lfs-fallback` remote. It will be used automatically if a
file can't be found on the regular remote.
Ref #137215
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/137615
This change adds `--prune-destructive` command line argument to the
`make_update.py` script which ensures directories which used to be
an in-worktree checkout do not exist. Currently it is addons and
addons_contrib folders.
This gives buildbot an ability to ensure checkout used for build is
correct, regardless of which branch or patch was built there.
The option is needed because sometimes it is still required to be
able to build patches against v330 or v360 which still use the old
style of addons.
Alternative solution could be to ignore the folders in the INSTALL
command, but this could have impact on users who wants to package
custom addons.
The downside of this solution is that buildbot potentially will be
downloading addons and addons_contrib repositories every time it
builds a patch against v330 or v360. It should not be a problem in
practice since the repositories are rather small, and patches for
these tracks are not built very often.
Ref #126557
Pull Request: https://projects.blender.org/blender/blender/pulls/127798
The `make update` will now always update all initialized submodules,
including the library ones.
The `make_update.py --no-libraries` will neither initialize nor update
pre-compiled libraries or tests.
Pull Request: https://projects.blender.org/blender/blender/pulls/118812
Solves the issue of the script potentially sitting for a long time
without having any progress reported. Confusingly, such behavior
depends on Git version.
In older versions (< 2.33) there will be progress reported, but it
then got changed by the commit in Git:
7a132c628e
The delayed checkout is exactly how Git LFS integrates into the Git
process. Another affecting factor for the behavior is that submodule
configured to use "checkout" update policy is forced to have quite
flag passed to the "git checkout":
https://github.com/git/git/blob/v2.43.2/builtin/submodule--helper.c#L2258
This is done to avoid the long message at the end of checkout about
the detached state of HEAD, with instructions how to resolve that.
There are two possible solutions: either use "rebase" update policy
for submodules, or skip Git LFS download during the submodule update.
Changing the update policy is possible, but it needs to be done with
a bit of care, and possible revised process for updating/merging
tests data.
This change follows the second idea of delaying LFS download for a
later step, so the process is the following:
- Run `git submodule update`, but tell Git LFS to not resolve the links
by using GIT_LFS_SKIP_SMUDGE=1 environment variable.
- Run `git lfs pull` for the submodule, to resolve the links.
Doing so bypasses hardcoded silencing in the Git. It also potentially
allows to recover from an aborted download process.
The `git lfs pull` seems to be a nominal step to resolve the LFS links
after the smudging has been skipped. It is also how in earlier Git
versions some Windows limitations were bypassed:
https://www.mankier.com/7/git-lfs-faq
The submodule update now also receives the "--progress" flag, which
logs the initial Git repository checkout process, which further
improves the feedback.
The byproduct of this change is that an error during precompiled
libraries and tests data update is not considered to be fatal.
It seems to be more fitting with other update steps, and allows
more easily reuse some code.
There is also a cosmetic change: the messages at the end of the
update process now have their own header, allowing more easily
see them in the wall-of-text.
Pull Request: https://projects.blender.org/blender/blender/pulls/118673
This change makes it so build system and update utilities for Blender builds
are using pre-compiled libraries and other resources attached as Git modules
instead of using checkout of SVN repositories in the parent folder.
The directory layout:
```
* release/datafiles/
* assets/ -> blender-assets.git
* publish/
* ...
* README.txt
* lib/
* darwin_x64/ -> lib-darwin_x64.git
* darwin_arm64/ -> lib-darwin_arm64.git
* linux_x64/ -> lib-linux_x64.git
* windows_x64/ -> lib-windows_x64.git
* tests/
* data/ -> blender-test-data.git
```
The changes about configuring the actual Git sub-modules are not included
into this patch, as those require repository to actually exist before it
can be used.
The assets submodule is enabled by default, and the rest of them are
disabled. This means that if someone runs `git submodule update --init`
they will not get heavy libraries. The platform-specific and tests
related submodules are enabled when using `make update` or `make test`.
All the submodules are tracked: this means that when new commits are
done to the submodule, the blender.git repository is to be updated to
point them to the new hash. This causes some extra manual work, but it
allows to more easily update Blender and its dependencies to known good
state when performing operations like bisect.
Ref #108978
Pull Request: https://projects.blender.org/blender/blender/pulls/117946
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Fix the issue when upstream workflow is used and the addons fork does
not yet have the release branch. In this case create a local branch
which is based on the upstream/<branch> but does not track anything.
Typically with such workflow the local branches will track origin, but
since the origin does not have the branch yet it is not possible to
track it.
Test plan:
- Use upstream workflow
- Have a fork of addons, which does not have blender-v3.6-release
- Run make update
The error message:
```
Updating scripts/addons ...
git fetch origin
git fetch upstream
git checkout -t origin/blender-v3.6-release
fatal: 'origin/blender-v3.6-release' is not a commit and a branch 'blender-v3.6-release' cannot be created from it
make: *** [update] Error 128
```
Pull Request: https://projects.blender.org/blender/blender/pulls/108197
- make_update.external_script_add_origin_if_needed
returned None or an empty string. As neither where checked simply
return None.
- git_get_remote_url had an incorrect annotation.
Apparently `git checkout -t` is only allowed to happen for new branches.
Added a code which checks whether the branch already exists and it so
uses the `git checkout <branch>`.
Pull Request #105234
This commit implements described in the #104573.
The goal is to fix the confusion of the submodule hashes change, which are not
ideal for any of the supported git-module configuration (they are either always
visible causing confusion, or silently staged and committed, also causing
confusion).
This commit replaces submodules with a checkout of addons and addons_contrib,
covered by the .gitignore, and locale and developer tools are moved to the
main repository.
This also changes the paths:
- /release/scripts are moved to the /scripts
- /source/tools are moved to the /tools
- /release/datafiles/locale is moved to /locale
This is done to avoid conflicts when using bisect, and also allow buildbot to
automatically "recover" wgen building older or newer branches/patches.
Running `make update` will initialize the local checkout to the changed
repository configuration.
Another aspect of the change is that the make update will support Github style
of remote organization (origin remote pointing to thy fork, upstream remote
pointing to the upstream blender/blender.git).
Pull Request #104755
rB546314fc9669 broke `svn_libraries_base_url` utils API compatibility
for no good reasons, making new `branch` argument explicitely optional.
Reported on chat by Michael Kowalski (@makowalski), thanks!
Needed for studio sprite-fright frozen branch.
Also do not overwrite branch for git sub-modules when it is defined, and
fallback to `master` branch in case specified branch is not found in a
specific sub-repository.
This prints a more informative message, and is convenient when working with
local changes or in a branch where you only need to update submodules or tests.
