RenderTest: GPU Render Tests Pass Silently

Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.

* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.

`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).

Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.

Pull Request: https://projects.blender.org/blender/blender/pulls/117629
This commit is contained in:
Jeroen Bakker
2024-01-29 15:39:14 +01:00
parent 7163873420
commit 324ef0cbf4
9 changed files with 37 additions and 13 deletions

View File

@@ -58,6 +58,7 @@ def create_argparse():
parser.add_argument("-outdir", nargs=1)
parser.add_argument("-oiiotool", nargs=1)
parser.add_argument('--batch', default=False, action='store_true')
parser.add_argument('--fail-silently', default=False, action='store_true')
return parser
@@ -80,7 +81,7 @@ def main():
if test_dir_name.startswith('hair') and platform.system() == "Darwin":
report.set_fail_threshold(0.050)
ok = report.run(test_dir, blender, get_arguments, batch=args.batch)
ok = report.run(test_dir, blender, get_arguments, batch=args.batch, fail_silently=args.fail_silently)
sys.exit(not ok)