On Windows an entire directory may be locked when any files inside it
are opened by another process. This can cause operations that
recursively remove a directory (uninstalling & updating) to fail
with a partially removed extension.
The case of uninstalling was already handled, where failure to remove
a directory would stage the extension for later removal.
In the case of updating however, the user could be left with a broken
(partially removed) extension where some files were removed, as the
directory was locked, the update would fail to extract new files.
Address this issue by renaming the directory before recursive removal.
The following logic has been implemented:
- If any files in the directory are locked, renaming will fail.
So even though the operation fails the extension is left intact.
- If renaming succeeds, it's possible to apply the update.
While it's possible (albeit unlikely) recursive removal fails,
which could be caused by file-system permissions issues corruption or
a process could open a file between rename & removal.
In this case the renamed directory is staged for later removal.
Other changes:
- Resolve a related problem where the user could install an
extension previously staged for removal, now installing an extension
ensured it's not removed later.
This would occur if uninstalling failed, the user resolves
directory-lock, uninstalls again, then re-installs the extension.
- When an extension fails to be removed, don't attempt to remove
user configuration for that extension.
Prefer to keep the extension & it's settings in their "current state"
if it can't be removed.
Do not report error when local repo does not have manifest and we are
asking for remote repositories.
If the errors list is filled with any error the operation is considered
failure.
Ref: !127360
Not sure why, but today I got almost systematic deadlocks in the
ProcessPoolExecutor calls (and similar issues with direct usage of
multiprocessing)... For now synchronous single process will do.
- Paths of C++-parsed files were not properly 'unixified' on Windows.
This was bad both for changes noisyness in PO files, and broke on
the un-escaping of `\n` and `\t` sequences.
- The `ProcessPoolExecutor` starts sub-processes differently on Linux
than on Windows or OSX. While Linux's `fork` keeps the same
environment (i.e. all Blender stuff remains available in workers
subprocesses), the 'spawn' used on Windows (and reportedly OSX) starts
a new bare python interpreter. This means that code executed by these
needs to be Blender-agnostic to be portable.
The only thing that is currently known broken on non-Linux platforms is
the RTL processing of some languages like Arabic or Hebrew.
Since [0] missing repository directories printed a warning on startup.
When checking for blocked extensions on startup, skip directories
that don't exist & don't print any warnings if the repository data
hasn't been downloaded.
[0]: 656fe6d3e4
When deleting files on WIN32, open files cannot be removed.
This is especially a problem for compiled Python modules which
remain open once imported.
Previously it was not as common for add-ons to include compiled Python
modules however with extensions supporting Python-wheels,
it's increasingly likely users run into this.
Workaround the problem by:
- Scheduling the files for removal next time Blender starts.
- Rename paths that cannot be removed to avoid collisions when
the paths is reused (re-installing for example).
This is supported for:
- Extensions.
- Python wheels.
- Legacy user add-ons.
- App-templates.
Details:
- On startup, a file exists that indicates cleanup is needed.
In the common case the file doesn't exist.
Otherwise module paths are scanned for files to remove.
- Since errors resolving paths to remove could result in user data loss,
ensure the paths are always within the (extension/addon/app-template)
directory.
- File locking isn't used, if multiple Blender instances start at the
same time and try to remove the same files, this won't cause errors.
Even so, remove the checking file immediately avoid unnecessary
file-system access overhead for other Blender instances.
Also resolves#125049.
Expose arguments to use when creating a Python sub-process.
Python could fail to start when loaded in a customized environment,
with PYTHONPATH set for e.g. Blender ignores these and loads but a
Python sub-process attempts to use these environment variables which
may point to incompatible Python versions.
Resolve the root cause of #124731.
A template string was accidentally turned into an f-string in
78b9218c98, which changes the way it is
evaluated. Easy enough to roll back that one change.
Workaround: `[ASN1] nested asn1 error` error when making HTTPS
connections on systems with certificates that OpenSSL cannot parse
are installed.
This is a general issue with Python, resolve by applying a proposed
fix [0] to the extensions Python process at run-time.
(this doesn't impact Blender's Python run-time).
The down side is HTTPS connections will only work for extensions
on systems with this problem so this needs to be resolved by Python
long term.
While any changes to Python's SSL checks is worth avoiding,
this simply skips SSL certificates in the windows store that OpenSSL
can't parse instead of failing all SSL connections.
See related issues:
- https://github.com/python/cpython/issues/79846
- https://github.com/openssl/openssl/issues/25023
[0]: https://github.com/python/cpython/pull/91740
Ref !124943.
