When fixing issues, seeing operation results can be helpful for
detecting which operation went wrong.
This commit adds an option for exporting all operations results to
image files.
Exceptions are:
- Output operations: They are already exported or can be seen in UI.
- Constant operations: There are too many and is rarely useful.
They are exported to "<temp session folder>/COM_operations/"
with filenames "<operation class name>_<operation id>.png".
Only works on full frame execution mode.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11722
Currently we mostly iterate buffer areas using x/y loops or through
utility methods extending from base classes.
To simplify code in simple operations this commit adds wrappers for
specifying buffer areas and their iterators for raw buffers with any
element stride:
- BufferRange: Specifies a range of contiguous buffer elements from a
given element index.
- BufferRangeIterator: Iterates elements in a BufferRange.
- BufferArea: Specifies a rectangle area of elements in a 2D buffer.
- BufferAreaIterator: Iterates elements in a BufferArea.
- BuffersIterator: Simultaneously iterates an area of elements in an
output buffer and any number of input buffers.
- BuffersIteratorBuilder: Helper for building BuffersIterator adding
buffers one by one.
For iterating areas coordinates it adds `XRange` and `YRange` methods
that return `IndexRange`.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11882
Not causing issues in current master because all buffer areas are at
(0, 0) position and `Extend` is not used. But areas may be at any
position in future developments and it will crash.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11784
Link sockets are always connected to inserted translate or scale
operation `Color` sockets even when they have different data type.
This causes crashes on full frame mode when operations read inputs
with non expected datatypes.
Because data type conversions need to be executed before, convert
resolutions must ensure same datatypes are linked.
Work scheduler needed initialization and execution models are
not created during constant folding. This moves work execution
method to execution system.
Graphs are usually large, needing a lot of horizontal scrolling and
they can include more information for debugging.
This patch makes graph more compact horizontally by splitting
labels in lines and removing namespaces.
Furthermore it adds following information:
- Operation ID.
- SetValueOperation float value.
- Optionally, operation node name.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11720
Currently there is no clear way to know if an operation is constant
(i.e. when all rendered pixels have same values). Operations may
need to get constant input values before rendering to determine
their resolution or areas of interest. This is the case of scale, rotate
and translate operations. Only "set operations" are known as
constant but many more are constant when all their inputs are so.
Such cases can be optimized by only rendering one pixel.
Current solution for tiled implementation is to get first pixel
from input. This works for root execution groups, others
need previous groups to be rendered.
On full frame implementation this is not possible, because buffers
are created on rendering to reduce peak memory and there is
no per pixel calls.
This patch evaluates all operations that are constant into primitive
operations (Value/Vector/Color) before determining resolutions.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11490
Adds full frame implementation to Image node operations.
Mostly refactored into buffer utility methods for reuse in other
operations.
No functional changes.
1.8x faster than tiled fallback.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11559
This patch fixes many minor spelling mistakes, all in comments or
console output. Mostly contractions like can't, won't, don't, its/it's,
etc.
Differential Revision: https://developer.blender.org/D11663
Reviewed by Harley Acheson
After looking into task isolation issues with Sergey, we couldn't find the
reason behind the deadlocks that we are getting in T87938 and a Sprite Fright
file involving motion blur renders.
There is no apparent place where we adding or waiting on tasks in a task group
from different isolation regions, which is what is known to cause problems. Yet
it still hangs. Either we do not understand some limitation of TBB isolation,
or there is a bug in TBB, but we could not figure it out.
Instead the idea is to use isolation only where we know we need it: when
holding a mutex lock and then doing some multithreaded operation within that
locked region. Three places where we do this now:
* Generated images
* Cached BVH tree building
* OpenVDB lazy grid loading
Compared to the more automatic approach previously used, there is the downside
that it is easy to miss places where we need isolation. Yet doing it more
automatically is also causing unexpected issue and bugs that we found no
solution for, so this seems better.
Patch implemented by Sergey and me.
Differential Revision: https://developer.blender.org/D11603
Changes introduced in commit rBe9f2f17e8518
can create different render results when there is
a Math or Mix operation after TextureOperation
on tiled execution model.
This is due to WriteBufferOperation forcing a single pixel
resolution when these operations use a preferred
resolution of 0 to check if their inputs have resolution.
Fixing this behaviour creates different renders too.
This patch keeps previous tiled implementation and
adds the new implementation only for full frame execution.
Reviewed By: Jeroen Bakker (jbakker)
Differential Revision: https://developer.blender.org/D11546
In order to reduce stack size this patch converts full frame
recursive methods into iterative.
- No functional changes.
- No performance changes.
- Memory peak may slightly vary depending on the tree because
now breadth-first traversal is used instead of depth-first.
Tests in D11113 have same results except for test1 memory peak:
360MBs instead of 329.50MBs.
Reviewed By: Jeroen Bakker (jbakker)
Differential Revision: https://developer.blender.org/D11515
Under some circumstances using task isolation can cause deadlocks.
Previously, our task pool implementation would run all tasks in an
isolated region. Now using task isolation is optional and can be
turned on/off for individual task pools.
Task pools that spawn new tasks recursively should never enable
task isolation. There is a new check that finds these cases at runtime.
Right now this check is disabled, so that this commit is a pure refactor.
It will be enabled in an upcoming commit.
This fixes T88598.
Differential Revision: https://developer.blender.org/D11415
This patch adds the base code needed to make the full-frame system work for both current tiled/per-pixel implementation of operations and full-frame.
Two execution models:
- Tiled: Current implementation. Renders execution groups in tiles from outputs to input. Not all operations are buffered. Runs the tiled/per-pixel implementation.
- FullFrame: All operations are buffered. Fully renders operations from inputs to outputs. Runs full-frame implementation of operations if available otherwise the current tiled/per-pixel. Creates output buffers on first read and free them as soon as all its readers have finished, reducing peak memory usage of complex/long trees. Operations are multi-threaded but do not run in parallel as Tiled (will be done in another patch).
This should allow us to convert operations to full-frame in small steps with the system already working and solve the problem of high memory usage.
FullFrame breaking changes respect Tiled system, mainly:
- Translate, Rotate, Scale, and Transform take effect immediately instead of next buffered operation.
- Any sampling is always done over inputs instead of last buffered operation.
Reviewed By: jbakker
Differential Revision: https://developer.blender.org/D11113
These variables and methods should make it easier to loop through buffers elements/pixels. They take into account single element buffers.
Single element buffers can be used for set operations to reduce memory usage.
Usage example: P2078
Reviewed By: #compositing, jbakker
Differential Revision: https://developer.blender.org/D11015
Initial report was mentioning the Classroom demo scene, but this is
probably because the scene was pre-configured to be used with OpenCL.
Would expect any OpenCL compositing to be failing prior to this fix.
The reason why crash was happening is due to OpenCL queue being
released from OpenCLDevice destructor. Is not that obvious, but
when Vector (including std::vector) is holding elements by value
a destructor will be called on "old" memory when vector capacitance
changes.
Solved by making forbidding copy semantic for compositor devices and
forcing move semantic to be used.
Also use emplace semantic in the devices vector initialization.
WorkScheduler task model deletes work packages after executing them. The other models don't do so. All models should handle packages the same way.
Reviewed By: #compositing, jbakker
Differential Revision: https://developer.blender.org/D11102
WorkScheduler task model deletes work packages after executing them. The other models don't do so. All models should handle packages the same way.
Reviewed By: #compositing, jbakker
Differential Revision: https://developer.blender.org/D11102
Some operations may use no preferredResolution ({0, 0}) when calling determineResolution on inputs to check if they have resolution on their own. See MixOperation or MathOperation determineResolution implementation. In such cases {0, 0} resolution ends up being set when an input doesn't have own resolution, breaking propagation of the original preferredResolution. They don't mean to set it as resolution, it's just a check.
This patch only allows to set valid resolutions (>0). When it's 0 it may be understood as "No preferred or determined resolution" so it should not be set to give output operations another chance of finding a proper resolution by calling determineResolution again with a different preferredResolution.
Test file:
{F9932526}
Reviewed By: #compositing, jbakker
Differential Revision: https://developer.blender.org/D10972
File output node always received the resolution from the first socket.
When that socket didn't had a link it would use a resolution of 0,0.
What lead to not saving the file at all.
This only effected Multi layer OpenEXR files.
This change would go over all the links to find the first valid
resolution.
This removes a lot of unnecessary code that is generated by
the compiler automatically.
In very few cases, a defaulted destructor in a .cc file is
still necessary, because of forward declarations in the header.
I removed some defaulted virtual destructors, because they are not
necessary, when the parent class has a virtual destructor already.
Defaulted constructors are only necessary when there is another
constructor, but the class should still be default constructible.
Differential Revision: https://developer.blender.org/D10911
For debugging purposes to convert the internal state of the
NodeOperationBuilder to a graphviz.
Usage:
std::cout << *this << "\n";
Inside any method of the NodeOperationBuilder.
