This allows exclusions to be set in the `tests` filter for the
benchmarks using the exclamation mark. Exclusions apply *after*
inclusions ie:
tests = ['*', '!victor', '!koro']
would select all available tests then specifically exclude victor
and koro, while:
tests = ['!victor']
would just exclude victor, since nothing is marked for inclusion
no tests will be run.
tests = ['!victor','bmw27,'victor']
would exclude victor, but include bmw27 and victor but as victor was
marked for exclusion only bmw27 will be ran.
Requested by bart in #146019
Pull Request: https://projects.blender.org/blender/blender/pulls/147588
Stored undo step data for position changes in sculpt mode are now
automatically compressed. Compression is run in background threads,
reducing memory consumption during sculpting sessions while adding
little performance overhead.
For testing and benchmarks, memory usage is now available through
`bpy.app.undo_memory_info()`. Undo memory usage is now tracked by the
existing automated benchmark tests. Some changes to the web benchmark
visualization present the data a bit better.
ZSTD compression is run asynchronously in a backround task pool.
Compression is only blocking if the data is requested immediately for
undo/redo.
Co-authored-by: Hans Goudey <hans@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/141310
This PR adds code for setting the Quality of Service (QoS) level of the
process on Windows. This can, e.g., make sure that on hybrid systems
P-cores are utilized even when the app window is out of focus.
In the following cases, it is adjusted from the default behavior:
- In wm_jobs.cc the QoS level is raised while a priority job is running.
- The command line option `--qos [high|eco|default]` can be used to
change the QoS level of the process.
- By default, the QoS level is raised for the EEVEE performance tests,
as they check viewport rendering performance and would otherwise be
reliant on never going out of focus to not get a downgraded QoS level.
By default, they are created with an out of focus window at the time
of landing this PR. This PR makes sure that they actually measure the
animation replay performance attainable during real-world use.
Pull Request: https://projects.blender.org/blender/blender/pulls/144224
When a benchmark test was failing, there was very little info available
to investigate it. Now report the stdout/stderr generated by the failing
command.
This commit gives users of the Cycles performance benchmark tool the
option to run performance benchmarks with OSL enabled for CPUs
and OptiX devices.
This can be done by adding `-OSL` to the device name:
`CPU-OSL`
`OPTIX-OSL_0`
Pull Request: https://projects.blender.org/blender/blender/pulls/136506
Similar to the regression tests it is possible to append -RT suffix
to the compute type to enable hardware ray-tracing.
For example HIP_0 will run benchmark on first HIP device without
hardware ray-tracing, HIP-RT_0 will run benchmark on the same device
but will enable hardware ray-tracing.
The downside of this change is that it will make it so METAL device
will no longer use HW-RT on M3 and aboce, and explicit METAL-RT is
to be used. This is because benchmark was relying on the Auto
configuration which has different behavior depending on the device
generation.
Pull Request: https://projects.blender.org/blender/blender/pulls/136308
This change makes it so build system and update utilities for Blender builds
are using pre-compiled libraries and other resources attached as Git modules
instead of using checkout of SVN repositories in the parent folder.
The directory layout:
```
* release/datafiles/
* assets/ -> blender-assets.git
* publish/
* ...
* README.txt
* lib/
* darwin_x64/ -> lib-darwin_x64.git
* darwin_arm64/ -> lib-darwin_arm64.git
* linux_x64/ -> lib-linux_x64.git
* windows_x64/ -> lib-windows_x64.git
* tests/
* data/ -> blender-test-data.git
```
The changes about configuring the actual Git sub-modules are not included
into this patch, as those require repository to actually exist before it
can be used.
The assets submodule is enabled by default, and the rest of them are
disabled. This means that if someone runs `git submodule update --init`
they will not get heavy libraries. The platform-specific and tests
related submodules are enabled when using `make update` or `make test`.
All the submodules are tracked: this means that when new commits are
done to the submodule, the blender.git repository is to be updated to
point them to the new hash. This causes some extra manual work, but it
allows to more easily update Blender and its dependencies to known good
state when performing operations like bisect.
Ref #108978
Pull Request: https://projects.blender.org/blender/blender/pulls/117946
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
There should be no functional changes for the typical usecase,
but it allows to have more tricky setups like pointing to a
BAT script to override some configuration.
The issue is that BAT scripts do not support new lines in the
command line arguments. That's where single-line python expression
helps.
For example, it is possible to point benchmark script to a blender.bat
which contains
blender.exe --python-expr "import bpy; bpy.context.preferences.addons['cycles'].preferences.use_oneapirt = False" %*
to have side-by-side numbers of oneAPI with and without HW RT.
Without this change the %* is which did not work: the BAT
script did not "see" part of the command line past the new line.
Pull Request: https://projects.blender.org/blender/blender/pulls/109006
The scene contains some interesting names, which requires to be
written as utf-8. And on Windows file descriptor is not guaranteed
to be using utf-8. Or, will error out if the invalid utf-8 sequence
is written.
This change makes it so running benchmarks on windows it fully successful.
Pull Request: https://projects.blender.org/blender/blender/pulls/108982
I noticed that sometimes the geometry nodes benchmark would not work
if there were some left-over debug prints in the code. The reason it did
not work was because the prints were mixed with the test output print.
I also tried using explicit flusing and `atexit` for printing the test output,
but that didn't solve it. Just printing additional newlines to better separate
the test output from other printed output helped though.
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
The test script did not work on windows
since it had some trouble importing the
api module on the blender side of things.
turning the file path to the module into
a raw string literal sidesteps the
backslash issue in the path.
Differential Revision: https://developer.blender.org/D13163
Reviewed by: brecht
* Make "run" command (re-)run all tests, add "update" command to only
run queued and outdated tests equivalent to the old "run" command.
* Support specifying environment variables for revisions, to easily
compare multiple parameter values.
* Better sorting of revisions in graph.
* Allow specifying a folder and automatically setting the proper executable
name depending on the operating system
* Use executables from configs for listing devices instead of a blender
command being available
The general graphing mechanism will create one graph for each output
variable. So it's not limited to time and memory, but that is what the
Cycles tests now output.
These are scripts for benchmarking Blender features on real-world .blend
files. They were originally written for benchmarking Cycles performance, and
were made generic so they can be used for more Blender features.
The benchmarks can be run locally by developers. But the plan is to also run
these as part of continuous integration to track performance over time.
Currently there are tests for Cycles rendering and .blend file loading.
Documentation:
https://wiki.blender.org/wiki/Tools/Tests/Performance
Main features:
* User created configurations to quickly run, re-run and analyze a selected
subset of tests.
* Supports both benchmarking with existing builds, and automatic building of
specified git commits, tags and branches.
* Generate HTML page with bar and line graphs from test results.
* Controlled using simple command line tool.
* For writing tests, convenient abstraction to run a Python function in Blender
with arguments and return value.
Ref T74730
Differential Revision: https://developer.blender.org/D11662
