Add two sets of options to the IO Report class that allows tests to
control the verbosity of output.
- Add `Report.context_lines` controlling how many lines of context the
diff uses when there are failures (3 by default)
- Add `Report.side_to_print_single_line` (5 by default) and
`Report.side_to_print_multi_line` (3 by default) controlling how many
items are written out
The first option helps when a failure in the test might not produce
enough lines of output to know which object is affected. The second set
of options allows individual tests to ensure more values are taken into
consideration for test validation.
They are exposed as class variables due to all the inner methods using
them being static.
Pull Request: https://projects.blender.org/blender/blender/pulls/143922
Add the new Curves datablock type to the IO report. This was created
for the new hair system but is more generally used for any type of curve
generated within geometry nodes.
The report itself is mostly just a dump of all available attributes for
the object.
Pull Request: https://projects.blender.org/blender/blender/pulls/142925
HDR video files are properly read into Blender, and can be rendered out
of Blender.
HDR video reading / decoding:
- Two flavors of HDR are recognized, based on color related video
metadata: "PQ" (Rec.2100 Perceptual Quantizer, aka SMPTE 2084) and
"HLG" (Rec.2100 Hybrid-Log-Gamma, aka ARIB STD B67). Both are read
effectively into floating point images, and their color space
transformations are done through OpenColorIO.
- The OCIO config shipped in Blender has been extended to contain
Rec.2100-PQ and Rec.2100-HLG color spaces.
- Note that if you already had a HDR video in sequencer or movie clip,
it would have looked "incorrect" previously, and it will continue to
look incorrect, since it already has "wrong" color space assigned to
it. Either re-add it (which should assign the correct color space),
or manually change the color space to PQ or HLG one as needed.
HDR video writing / encoding"
- For H.265 and AV1 the video encoding options now display the HDR mode.
Similar to reading, there are PQ and HLG HDR mode options.
- Reference white is assumed to be 100 nits.
- YUV uses "full" ("PC/jpeg") color range.
- No mastering display metadata is written into the video file, since
generally that information is not known inside Blender.
More details and screenshots in the PR.
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/120033
This is a basic armature deformation test for #141535 using Lattices instead of
Mesh as the target object type. Lattice deformation was briefly broken, which is
caught by this test.
The test adds the general-purpose `unit_test_compare` function to lattice object
data. It only compares lattice point counts and positions for now, more data can
be added later if necessary.
The `MeshTest` class did not support lattice object types yet, so needed some
changes. The Curves case was already supported, but only by full conversion to
mesh data, without actually using the `unit_test_compare` function specific for
curves geometry. This is unchanged, because applying constructive modifiers on
curves does not work. If it were not for this limitation the test could do
actual curves comparisons now.
For lattice support the `MeshTest` class comparison function has been
generalized to all supported object data types. It runs the appropriate
`unit_test_compare` api function and validation where supported (only meshes at
this point).
Pull Request: https://projects.blender.org/blender/blender/pulls/141546
Test for armature deform modifier settings that are not yet covered by other tests:
- Vertex Group vs. Envelope deformation and both combined.
- Vertex group masking ('vertex_group' setting of the modifier)
- Inverted vertex group masking
- Preserve Volume (dual quaternions)
- Vertex Group/Envelope influence mixing (bone option)
- B-Bone deformation
- Multi-modifier mixing
Each case has a new test/expected mesh pair in the modifiers test.
Pull Request: https://projects.blender.org/blender/blender/pulls/141054
The `apply_modifiers` property of the `RunTest` class overrides
all of the test level `apply_modifier` properties. This prevents
modifiers from manually specifying when a modifier is applied and forces
the modifier to be applied immediately after it is added.
The vast majority of tests do not override the `apply_modifier`
property, the primary usecase for this property is to work in
combination with the `do_compare` property to allow examining the
corresponding .blend file to debug test failures.
This commit simplifies the settings by removing this parameter. It now
only disables applying the modifier if `do_compare` is set to False.
Pull Request: https://projects.blender.org/blender/blender/pulls/140893
The 2D->2D, 3D->3D, 4D->4D hash functions used in Voronoi node were
using quite an expensive hash function. Switch these to dedicated
2D/3D/4D hash functions (pcg2d, pcg3d, pcg4d) -- these are still very
good quality, but the hash function itself is 3x-4x faster.
Which makes Voronoi node calculation overall be around 2x faster. In
some cases when using OSL, the speedup is even larger.
This visibly changes output of the Voronoi noise however. The actual
noise "behaves" the same, just if someone was depending on the noise
pattern being exactly like it was before, this will change the pattern.
Images, more performance results and details wrt OSL are in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/139520
When opening Geometry Nodes regression test files I always find a bit annoying
that the `expected_object` and not the `test_object` is active. That's because
only the `test_object` contains the modifier or node tree that is being tested.
Therefore, usually my first step when opening these files is to select the
`test_object` first.
This patch changes it so that when mesh tests are updated, the `test_object` is
made active in the end before the file is saved again.
Pull Request: https://projects.blender.org/blender/blender/pulls/139088
- FBX "root bone" should become the Armature object itself, and not
an extra bone (follow same logic as Python importer did).
- "World to armature matrix" was not correct for armatures that are
parented under some other objects with transforms.
- Parenting imported meshes under an Armature was not taking into
account that the mesh bind transform might not be the same as the
current mesh node transform (i.e. was not setting "matrix parent
inverse" to compensate like the Python importer did).
- The repro file in #137768 also exposed an issue that importing custom
vertex normals was not working correctly in the new importer, when
mesh is partially invalid (validation alters the mesh, custom normals
have to be set afterwards).
Pull Request: https://projects.blender.org/blender/blender/pulls/138736
This change moves the tests data files and publish folder of assets
repository to the main blender.git repository as LFS files.
The goal of this change is to eliminate toil of modifying tests,
cherry-picking changes to LFS branches, adding tests as part of a
PR which brings new features or fixes.
More detailed explanation and conversation can be found in the
design task.
Ref #137215
Pull Request: https://projects.blender.org/blender/blender/pulls/137219
- Object parent type & parent bones
- Pose information that is not trivial (i.e. print posed bones that
have either non-identity pose matrix, or custom properties)
- Make sure custom properties are output sorted by name
Pull Request: https://projects.blender.org/blender/blender/pulls/136321
`oiiotool` diff generation is single-threaded and can become a
bottleneck on fast to render tests.
This PR generates diffs in parallel using the `multiprocessing` module.
Overlay tests (local): 90s -> 30s
buildbot +gpu:
macOS: 2020s -> 1808s
Linux: 1901s -> 1327s
Pull Request: https://projects.blender.org/blender/blender/pulls/134938
This adds support for Overlay tests.
There are some differences with how we handle tests for other engines:
- The renders are captured using `bpy.ops.render.opengl()`, but this
won't work on our GPU build bots.
- A single blend file can run multiple tests by outputting a txt list
with the test names.
- Each overlay test blend file requires a matching script file with
the same name inside `tests/python/overlay/`.
- To reproduce a specific test state you can run
`blender "(...)/tests/data/overlay/<test>.blend" -P "(...)/tests/python/overlay/<test>.py" -- --test <test-number>`.
Note:
The current test permutations are WIP, so reference images are not
committed to the data repo for now.
Pull Request: https://projects.blender.org/blender/blender/pulls/133879
In ea7d07098b, the RenderReport base
class was reworked to work better with test variations.
This commit reworks the Cycles render tests file to use the new
variation system for Hardware Ray tracing and OSL tests, avoiding the
custom system put in place in previous commits.
Pull Request: https://projects.blender.org/blender/blender/pulls/133813
When a given render test defined in CMakeLists.txt with a `--outdir`
parameter ends with a trailing slash, the resulting global report
overwrites the specific test's report. This is because `os.path.dirname`
for a path that ends in a slash returns the same directory, for example,
`os.path.dirname('foo/bar/') => 'foo/bar'
To avoid tests being able to put the report into a weird state, this
commit normalizes the `--outdir` path to strip trailing slashes.
Pull Request: https://projects.blender.org/blender/blender/pulls/133791
This commit reworks the RenderReport base class to avoid adding
`--cycles-device` device arguments to non Cycles tests.
This reduces some warnings that can show up with EEVEE and
Workbench tests that accidentally used these arguments.
Pull Request: https://projects.blender.org/blender/blender/pulls/133724
This commit checks the total number of files being processed for
rendering tests and causes it to emit a failure if none are found. This
has the benefit of loudly failing if there is a configuration error
instead of succeeding.
Pull Request: https://projects.blender.org/blender/blender/pulls/133691
Makes use of recently added test data to ensure proper handling of child
objects, with animated transform constraints, parented to other objects
who also have animated transform constraints.
Also uses the `colored_print` module to better segment the test output.
Pull Request: https://projects.blender.org/blender/blender/pulls/133600
- Animation keyframes now contain left/right handle information
- More fcurves are emitted
- Material textures emit whether their colorspace is "data"
- More files in fbx test coverage
Pull Request: https://projects.blender.org/blender/blender/pulls/133526
"Expected textual data output" comparison based tests for FBX,
OBJ, PLY, STL import.
- There's a tests/python/modules/io_report.py that can produce
a "fairly short text description of the scene" (meshes, objects,
curves, cameras, lights, materials, armatures, actions, images).
About each object, it lists some basic information (e.g. number
of vertices in the mesh), plus a small slice of "data" (e.g.
first few values of each mesh attribute).
- Custom import parameters, if needed, can be provided by
having a sidecar .json file next to imported file (same
basename, json extension), that would have a single json
object with custom arguments.
- Add FBX test coverage, with 46 fairly small files (total size 3.8MB)
covering various possible cases (meshes, animations, materials,
hierarchies, cameras, etc. etc.).
- Switch OBJ/PLY/STL import tests to the above machinery, remove C++
testing code.
Pull Request: https://projects.blender.org/blender/blender/pulls/132624
This adds a `unit_test_compare` function to `Curves`.
Compares the curves. Curves are the same if:
* The number of points matches.
* The number of curves matches.
* The attribute names are the same and have the same type.
* The point attribute values are within the `threshold`.
* The curve topology matches (e.g. the curve sizes are the same).
* The curve attribute values are within the `threshold`.
* The indices of the points and curves are the same (can be ignored if this should be treated as the same).
The implementation reuses the same functions as the existing
comparison function for meshes.
Pull Request: https://projects.blender.org/blender/blender/pulls/131164
BaseException was used as a catch-all in situations where it
didn't make sense and where "Exception" is more appropriate
based on Python's documentation & error checking tools,
`pylint` warns `broad-exception-caught` for e.g.
BaseException includes SystemExit, KeyboardInterrupt & GeneratorExit,
so unless the intention is to catch calls to `sys.exit(..)`,
breaking a out of a loop using Ctrl-C or generator-exit,
then it shouldn't be used.
Even then, it's preferable to catch those exceptions explicitly.
This commit adds a new cmake variable `WITH_CYCLES_TEST_OSL`
that runs every Cycles test a second time with OSL enabled.
At the moment only CPU OSL is enabled. There are plans to enable
OptiX OSL in the future when stability issues with OptiX OSL
have been resolved.
Some render tests have been blocked from running until we can figure
out a fix. The most notiable being all the Pricincipled BSDF tests
as some of them are failing due to noise differences.
Ref #123012
Pull Request: https://projects.blender.org/blender/blender/pulls/124601
Add a option to ignore Cycles render test black list, controlled via
BLENDER_TEST_IGNORE_BLOCKLIST environment variable.
Useful for testing to see if anything is seriously broken in a test
that's black listed. (E.g. See if some particle rendering tests are
seriously broken on the GPU)
It would be recommened that this be used to test GPU compiler updates.
Or the enablement of certain features like custom curve rendering on
custom BVH.
Pull Request: https://projects.blender.org/blender/blender/pulls/124662
Currently the mesh boolean code is producing a different index order on the latest
version of XCode on macOS. This is proving very difficult to investigate, and it's
preventing developers from spending time on more important things.
As a compromise for now, allow configuring certain geometry nodes tests to make
meshes with a different index order still pass the test. This can be done by adding
a boolean custom property with the name `allow_index_change` to the test object
and setting it to "True". Allow the index change on different platforms is unfortunate,
spending time on working on a replacement exact boolean algorithm is a more
valuable use of time currently.
After this commit lands, someone who can reproduce the failing test on their
computer should update tests to include that custom property as necessary.
Pull Request: https://projects.blender.org/blender/blender/pulls/123204
Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.
* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.
`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).
Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.
Pull Request: https://projects.blender.org/blender/blender/pulls/117629
When running the render test for EEVEE-Next the command printed on
the report to update the reference images was incorrect. In stead of
displaying
`BLENDER_TEST_UPDATE=1 ctest -R eevee_next` it displayed
`BLENDER_TEST_UPDATE=1 ctest -R eevee next`.
The cause of this is that the title of the report is used to create
the command. EEVEE-Next has a space in its title which generates
an incorrect command.
A quick fix would be to replace spaces with underscores. But it would
be better to fix this more clearly by adding an attribute containing the
engine name.
This PR adds a `set_engine_name` method to the Report class. By
default the engine name is set based on the title.
Pull Request: https://projects.blender.org/blender/blender/pulls/117503
