This patch rewrites the Inpaint node in the Realtime Compositor. The old
method suffered from discontinuities and singularities in the inpainting
regions. Furthermore, it ignored semi-transparent areas.
The new method is inspired by a two pass method described by the paper:
Rosner, Jakub, et al. "Fast GPU-based image warping and inpainting for
frame interpolation." International Conferences on Computer Graphics,
Vision and Mathematics. 2010.
In particular, we first fill the inpainting region using jump flooding,
then we apply a variable size blur pass whose size is proportional to
the distance to the inpainting boundary. The smoothed region is then
mixed with the input using its alpha.
The new method is much closer to the Bertalmio-style diffusion-based
inpainting methods, and thus can more accurately close holes than
existing methods.
The aforementioned method requires variable size blur, which is quite
expensive for this use case, so a new implementation was added that
approximates the method using a separable implementation, which provides
a visually pleasing result assuming a sufficiently smooth radius field,
which is true for our case since the field is an SDF.
Fixes: #114422
Pull Request: https://projects.blender.org/blender/blender/pulls/114849
This patch adds support for full precision compositing for the Realtime
Compositor. A new precision option was added to the compositor to change
between half and full precision compositing, where the Auto option uses
half for the viewport compositor and the interactive render compositor,
while full is used for final renders.
The compositor context now need to implement the get_precision() method
to indicate its preferred precision. Intermediate results will be stored
using the context's precision, with a number of exceptions that can use
a different precision regardless of the context's precision. For
instance, summed area tables are always stored in full float results
even if the context specified half float. Conversely, jump flooding
tables are always stored in half integer results even if the context
specified full. The former requires full float while the latter has no
use for it.
Since shaders are created for a specific precision, we need two variants
of each compositor shader to account for the context's possible
precision. However, to avoid doubling the shader info count and reduce
boilerplate code and development time, an automated mechanism was
employed. A single shader info of whatever precision needs to be added,
then, at runtime, the shader info can be adjusted to change the
precision of the outputs. That shader variant is then cached in the
static cache manager for future processing-free shader retrieval.
Therefore, the shader manager was removed in favor of a cached shader
container in the static cache manager.
A number of utilities were added to make the creation of results as well as
the retrieval of shader with the target precision easier. Further, a
number of precision-specific shaders were removed in favor of more
generic ones that utilizes the aforementioned shader retrieval
mechanism.
Pull Request: https://projects.blender.org/blender/blender/pulls/113476
The compositor sometimes produces straight alpha even though
premultiplied alpha is expected. Moreover, there is an inconsistency
between the CPU and GPU compositors.
For the GPU compositor, this is because GPU textures sometimes store
straight alpha, while the compositor always expects premultiplied alpha,
so we need to premultiply the alpha in those cases.
For the CPU compositor, this is because the image operation didn't
premultiply the alpha of byte textures, so we need to ensure
premultiplied alpha in those cases.
There is a data loss issue in case of byte images, since the IMB module
unpremultiplies premultiplied images then the compositor premultiplies
it again. But this will be handled in a different patch since it require
some design and refactoring first.
Pull Request: https://projects.blender.org/blender/blender/pulls/114305
This patch changes how wrapped translations are handled by the Realtime
Compositor. Previously, translations were always stored on the result
and delayed until automatically realized later. The wrapping status was
also stored to control this later automatic realization.
This patch changes that such that translations are immediately realized
for the axes that has enabled wrapping. Consequently, the image will not
get translated, but its content will, in a clip on one side, wrap on the
opposite side manner.
Another change is that wrapping information is no longer propagated to
future automatic realizations, so tilling or repeating an image is no
longer possible. An alternative method of repetition will be introduced
in a later patch.
Pull Request: https://projects.blender.org/blender/blender/pulls/113669
Blender freezes when adding a Cryptomatte node and using the GPU
compositor, even if the node is later removed.
This was because the CPU compositor acquired a render result mutex and
never released it due to an early exit that missed that release. This
patch fixes that by adding an appropriate release in that early exist.
Translations in the realtime compositor are doubled where there is a
rotation or scale component.
That was due to applying translations even after domain realization. So
this patch restricts the translation to the case where domain
realization doesn't take place.
The GPU compositor crops the viewed images to the render resolution.
While the original size and content of the input to the viewer should be
retained as is.
This patch fixes that by specializing compositors that can use composite
outputs to be able to view images of any arbitrary size. This is still
missing the translation offset of the viewer, but this shall be tackled
separately.
This patch immediately realizes the scale and rotation components of
transformations at the point of transform nodes. The translate component is
still delayed and only realized when really needed to avoid clipping.
Transformed results are always realized in an expanded domain that avoids
clipping due to rotation or scaling. The size of the transformed domain is
clipped to the GPU texture size limit for now until we have support for huge
textures, that limit is typically 16k.
A potential optimization is to join all consecutive transform and realize
operations into a single realize operation.
Fixes#112332.
Pull Request: https://projects.blender.org/blender/blender/pulls/112332
This replaces the older dynamic c arrays with blender::Vector as
appropriate. Many files required minimal changes and the before/after
are quite similar.
There's 3 remaining usages of the old machinery but those will require
more involved changes and design.
See #103343
Pull Request: https://projects.blender.org/blender/blender/pulls/110981
This patch implements the Keying Screen node for the Realtime
Compositor. Draw data was introduced to the Movie Clip ID to allow
caching of the keying screen.
Pull Request: https://projects.blender.org/blender/blender/pulls/113055
This patch changes the interpolation algorithm utilized by the Keying
Screen node to a Gaussian Radial Basis Function Interpolation. This is
proposed because the current Voronoi triangulation based interpolation
has the following properties:
- Not temporally stable since the triangulation can abruptly change as
tracking markers change position.
- Not smooth in the mathematical sense, which is also readily visible in
the artists sense.
- Computationally expensive due to the triangulation and naive
rasterization algorithm.
On the other hand, the RBF interpolation method is temporally stable and
continuous, smooth and infinitely differentiable, and relatively simple
to compute assuming low number of markers, which is typically the case
for keying screen objects.
This breaks backward compatibility, but the keying screen is only used
as a secondary input for keying in typical compositor setups, so one
should expect minimal difference in outputs.
Pull Request: https://projects.blender.org/blender/blender/pulls/112480
This patch changes the image type used in the Jump Flooding Algorithm to
be Int2 instead of Float4. That's because we used to store the distance
along with the texel location, which we no longer do, so we are left
with the 2D texel location only which can be stored in an Int2 image.
We no longer store the distance because it is not necessarily needed, it
introduces a sqrt in each of the JFA passes, and it is less precise due
to storage in 16F images. Developers should compute the distance in the
user shader instead.
This is a non-functional change, but results in less memory usage,
higher performance, and higher precision.
Pull Request: https://projects.blender.org/blender/blender/pulls/112941
Previously, the Result class was reserved for inputs and outputs of
operations, so its allowed types were naturally those exposed to the
user. However, we now use the Result class internally for intermediate
results, so it now makes sense to expend the allowed types.
The types are now divided into two categories, those that are user
facing and need to be handled in implicit operations and those that
are internal and can be exempt from such handling. Internal types are
reserved for texture results, as the single value mechanism is only
useful for user facing results.
The patch merely adjusts the switch cases across the code base, adding
one new internal type as an example.
Pull Request: https://projects.blender.org/blender/blender/pulls/112414
This patch implements the Inpaint node for the Realtime Compositor. The
inpainting region is filled by sampling the color of the nearest boundary pixel
if it is not further than the user supplied distance. Additionally, a lateral
blur is applied in the tangential path to the inpainting boundary to smooth out
the inpainted region.
The implementation is not identical to the existing CPU implementation due to
technical infeasibility. In particular, the CPU implementation uses a Manhattan
distance transform, while the GPU implementation uses an Euclidean one, which is
a consequence of the use of the Jump Flooding algorithm. Furthermore, the CPU
uses a serial convolution starting from the boundary outwards, while the GPU
uses a lateral Gaussian blur in the direction tangent to the boundary.
Pull Request: https://projects.blender.org/blender/blender/pulls/111792
This patch implements the Double Edge Mask node for the Realtime
Compositor. The implementation is primarily based on the 1+JFA Jump
Flooding algorithm, which was also introduced in this commit.
Pull Request: https://projects.blender.org/blender/blender/pulls/112223
There are a couple of functions that create rna pointers. For example
`RNA_main_pointer_create` and `RNA_pointer_create`. Currently, those
take an output parameter `r_ptr` as last argument. This patch changes
it so that the functions actually return a` PointerRNA` instead of using
the output parameters.
This has a few benefits:
* Output parameters should only be used when there is an actual benefit.
Otherwise, one should default to returning the value.
* It's simpler to use the API in the large majority of cases (note that this
patch reduces the number of lines of code).
* It allows the `PointerRNA` to be const on the call-site, if that is desired.
No performance regression has been measured in production files.
If one of these functions happened to be called in a hot loop where
there is a regression, the solution should be to use an inline function
there which allows the compiler to optimize it even better.
Pull Request: https://projects.blender.org/blender/blender/pulls/111976
