This commit reworks the RenderReport base class to avoid adding
`--cycles-device` device arguments to non Cycles tests.
This reduces some warnings that can show up with EEVEE and
Workbench tests that accidentally used these arguments.
Pull Request: https://projects.blender.org/blender/blender/pulls/133724
This commit checks the total number of files being processed for
rendering tests and causes it to emit a failure if none are found. This
has the benefit of loudly failing if there is a configuration error
instead of succeeding.
Pull Request: https://projects.blender.org/blender/blender/pulls/133691
Makes use of recently added test data to ensure proper handling of child
objects, with animated transform constraints, parented to other objects
who also have animated transform constraints.
Also uses the `colored_print` module to better segment the test output.
Pull Request: https://projects.blender.org/blender/blender/pulls/133600
- Animation keyframes now contain left/right handle information
- More fcurves are emitted
- Material textures emit whether their colorspace is "data"
- More files in fbx test coverage
Pull Request: https://projects.blender.org/blender/blender/pulls/133526
"Expected textual data output" comparison based tests for FBX,
OBJ, PLY, STL import.
- There's a tests/python/modules/io_report.py that can produce
a "fairly short text description of the scene" (meshes, objects,
curves, cameras, lights, materials, armatures, actions, images).
About each object, it lists some basic information (e.g. number
of vertices in the mesh), plus a small slice of "data" (e.g.
first few values of each mesh attribute).
- Custom import parameters, if needed, can be provided by
having a sidecar .json file next to imported file (same
basename, json extension), that would have a single json
object with custom arguments.
- Add FBX test coverage, with 46 fairly small files (total size 3.8MB)
covering various possible cases (meshes, animations, materials,
hierarchies, cameras, etc. etc.).
- Switch OBJ/PLY/STL import tests to the above machinery, remove C++
testing code.
Pull Request: https://projects.blender.org/blender/blender/pulls/132624
This adds a `unit_test_compare` function to `Curves`.
Compares the curves. Curves are the same if:
* The number of points matches.
* The number of curves matches.
* The attribute names are the same and have the same type.
* The point attribute values are within the `threshold`.
* The curve topology matches (e.g. the curve sizes are the same).
* The curve attribute values are within the `threshold`.
* The indices of the points and curves are the same (can be ignored if this should be treated as the same).
The implementation reuses the same functions as the existing
comparison function for meshes.
Pull Request: https://projects.blender.org/blender/blender/pulls/131164
BaseException was used as a catch-all in situations where it
didn't make sense and where "Exception" is more appropriate
based on Python's documentation & error checking tools,
`pylint` warns `broad-exception-caught` for e.g.
BaseException includes SystemExit, KeyboardInterrupt & GeneratorExit,
so unless the intention is to catch calls to `sys.exit(..)`,
breaking a out of a loop using Ctrl-C or generator-exit,
then it shouldn't be used.
Even then, it's preferable to catch those exceptions explicitly.
This commit adds a new cmake variable `WITH_CYCLES_TEST_OSL`
that runs every Cycles test a second time with OSL enabled.
At the moment only CPU OSL is enabled. There are plans to enable
OptiX OSL in the future when stability issues with OptiX OSL
have been resolved.
Some render tests have been blocked from running until we can figure
out a fix. The most notiable being all the Pricincipled BSDF tests
as some of them are failing due to noise differences.
Ref #123012
Pull Request: https://projects.blender.org/blender/blender/pulls/124601
Add a option to ignore Cycles render test black list, controlled via
BLENDER_TEST_IGNORE_BLOCKLIST environment variable.
Useful for testing to see if anything is seriously broken in a test
that's black listed. (E.g. See if some particle rendering tests are
seriously broken on the GPU)
It would be recommened that this be used to test GPU compiler updates.
Or the enablement of certain features like custom curve rendering on
custom BVH.
Pull Request: https://projects.blender.org/blender/blender/pulls/124662
Currently the mesh boolean code is producing a different index order on the latest
version of XCode on macOS. This is proving very difficult to investigate, and it's
preventing developers from spending time on more important things.
As a compromise for now, allow configuring certain geometry nodes tests to make
meshes with a different index order still pass the test. This can be done by adding
a boolean custom property with the name `allow_index_change` to the test object
and setting it to "True". Allow the index change on different platforms is unfortunate,
spending time on working on a replacement exact boolean algorithm is a more
valuable use of time currently.
After this commit lands, someone who can reproduce the failing test on their
computer should update tests to include that custom property as necessary.
Pull Request: https://projects.blender.org/blender/blender/pulls/123204
Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.
* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.
`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).
Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.
Pull Request: https://projects.blender.org/blender/blender/pulls/117629
When running the render test for EEVEE-Next the command printed on
the report to update the reference images was incorrect. In stead of
displaying
`BLENDER_TEST_UPDATE=1 ctest -R eevee_next` it displayed
`BLENDER_TEST_UPDATE=1 ctest -R eevee next`.
The cause of this is that the title of the report is used to create
the command. EEVEE-Next has a space in its title which generates
an incorrect command.
A quick fix would be to replace spaces with underscores. But it would
be better to fix this more clearly by adding an attribute containing the
engine name.
This PR adds a `set_engine_name` method to the Report class. By
default the engine name is set based on the title.
Pull Request: https://projects.blender.org/blender/blender/pulls/117503
The part of the patch wasn't properly applied from a working branch,
and it wasn't very visible because the development environment already
had all files on disk.
Pull Request: https://projects.blender.org/blender/blender/pulls/117504
This change fixes confusion situation when the render output
is an RGBA image: the difference in color was not visible in
the report because alpha channel was all zeros. This is due
to idiff performing per-channel difference.
The solution to this problem is to have separate images for
color and alpha difference, which makes it clear where the
difference actually is coming from.
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Geometry nodes supports simulation nodes which allows later frames to depend
on previous frames. The existing geometry nodes regression tests only evaluated
the node tree at a single frame and therefore couldn't test the correct behavior of
simulations.
This adds a new kind of regression test that evaluates the scene at multiple
consecutive frames and then checks if the last frame matches.
Pull Request: https://projects.blender.org/blender/blender/pulls/109046
Recent failures requiring investigation have exposed some shortcomings
that this addresses:
- When creating the diff image for offline comparison, use a higher
threshold to prevent idiff from printing more output which will often
contradict the primary failure output just above it (very confusing)
- For metadata failures, make sure these get printed so it's obvious
what kind of failure we're dealing with
Pull Request: https://projects.blender.org/blender/blender/pulls/107058
This does 2 things to address the ARM64 failures:
- Increases the threshold to be inline with what Cycles uses
- Disables the 2 problematic WebP variations (#105006 will track)
This adds saving and loading tests for our supported image formats.
**Saving - bf_imbuf_save.py**
There are 2 template images which are loaded anew for each file save
attempt. One is an 8-bit RGBA image and the other 32-bit. This is
required as many formats use a variety of factors to determine which of
`ibuf->rect` or `ibuf->rectfloat` to use for processing. The templates
are constructed to have alpha transparency as well as values > 1 (or
clamped to 1 for the case of the 8-bit template).
Test flow:
- Load in an appropriate template image
- Save it to the desired format with the desired set of options
- Compare against the reference image
Notes:
- 98 references are used totaling ~3.6MB
- 10-12 second test runtime
- Templates can be reconstructed with the create-templates.blend file
**Loading - bf_imbuf_load.py**
Test flow:
- Load in each of the reference images
- Save them back out as .exr
- Save additional metadata to a secondary file (alpha mode, colorspace etc)
- Compare the saved out .exr with another set of reference .exrs
- Compare the saved out file metadata with set of reference metadata
Notes:
- 98 exr references are used totaling ~10MB
- 10-12 second test runtime as well
A HTML report is not implemented. The diff output organization is very
similar to the other tests so it should be somewhat easy to do in the
future if we want.
The standard set of environment variables are implemented for both:
BLENDER_TEST_UPDATE, BLENDER_VERBOSE, and BLENDER_TEST_COLOR
Pull Request #104442
Reference images in the reference_override_dir will be chosen before
images in reference_dir. This allows platform specific reference
images, with a common base.
Ignored when set to None. The caller is responsible
of setting the reference override dir as the unit test is more aware
what the definition of a platform is.
Patch adds `gpu.platform.device_type_get` function to get the device
type that blender has detected.
Reviewed By: brecht
Maniphest Tasks: T99046
Differential Revision: https://developer.blender.org/D15265
The old text was suggesting to run `BLENDER_TEST_UPDATE=1 ctest` for
failed tests. Now it's more clear that this is for the regeneration of
reference (ground truth) images, and that it will not touch passing test
cases.
It now also mentions to commit the new reference images to SVN, driving
the point home that this is for updating those, and not for making
failing tests succeed in general.
Over-the-shoulder reviewed by: @sergey
It can be assumed that all scripts comply with basic pep8 formatting
regarding white-space, indentation etc.
Also remove note in best practices page & update `tests/python/pep8.py`.
If we want to exclude some scripts from make format,
this can be done by adding them to `ignore_files` in:
source/tools/utils_maintenance/autopep8_format_paths.py
Or using `# nopep8` for to ignore for individual lines.
Ref T98554
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
