As part of PR #118396 the "generator" effect strips (color, text,
adjustment) also started to put their "raw" images into the cache.
However, seq_cache_timeline_frame_to_frame_index that has optimization
that tries to use only one cache entry based on how many frames are
present in the strip was not expecting to see effect strips, ever.
As a result, something like adjustment layer would effectively just
cache a single frame, since it has no "source frames" to speak of.
Fixed that by only doing this optimization for non-effect strips.
Ultimately this was due to missing nullptr check before trying to
process the image, but this should have been caught when loading
ImBufAnims. If any is missing, cancel multiview loading and load
movie as if it was single view only.
Original code was very confusing, with seemingly indecisive logic.
I hope, that it is less confusing now, however there is still some
indecisiveness present. This is because multiview configuration may not
be correct, so there must be fallback and only some multiview settings
require loading of multiple files. Perhaps be better wording could be
used for `open_anim_file_multiview()`.
Fortunately this is now expressed within 5 lines of code instead of 100.
Further this can be improved by checking if the file exists, since now
VSE crashes when right eye image is missing, because loading code
does not try to actually open the file.
Pull Request: https://projects.blender.org/blender/blender/pulls/118580
Often in previs setups, you have several "variations" of image/movie strips
for review, where the topmost one completely covers the others under it. VSE
rendering already had an optimization where if there's a fully opaque strip
that covers the whole screen, then the lower strips are skipped from
processing/rendering. However, it was not handling the case of non-fullscreen
strips (e.g. you'd have a Color strip near top & bottom for letterboxing, and
an opaque strip "in the middle").
This adds a simple "occluder tracking", and skips strips that are completely
covered by any single opaque strip that is above it (like outlined in #117790
task).
Playback of Gold previs between 1:42 and 1:55, on Windows/Ryzen5950X:
- Average frame time 28.5ms -> 23.8ms
- Median frame time 24.1ms -> 21.5ms
- Two slowest frames: 263->189ms, 194->178ms
Rendering the Gold previs movie: 325s -> 304s (93% of previous time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118396
The depsgraph CoW mechanism is a bit of a misnomer. It creates an
evaluated copy for data-blocks regardless of whether the copy will
actually be written to. The point is to have physical separation between
original and evaluated data. This is in contrast to the commonly used
performance improvement of keeping a user count and copying data
implicitly when it needs to be changed. In Blender code we call this
"implicit sharing" instead. Importantly, the dependency graph has no
idea about the _actual_ CoW behavior in Blender.
Renaming this functionality in the despgraph removes some of the
confusion that comes up when talking about this, and will hopefully
make the depsgraph less confusing to understand initially too. Wording
like "the evaluated copy" (as opposed to the original data-block) has
also become common anyway.
Pull Request: https://projects.blender.org/blender/blender/pulls/118338
This is different from applying effects to regular "source media" strips
like images or movies; an effect "hides" the input strip when it is
below itself in the channel.
Apply the same treatment to Color & Text strips. Fixes#118213.
If someone wants previous behavior for some reason, they can drag
the effect applied on Color/Text to be below the input strip.
Pull Request: https://projects.blender.org/blender/blender/pulls/118269
In some/many cases, an `ImBuf` is allocated, and all the pixels are
immediately filled by some code. Doing the memory clear within allocation
is just memory traffic for no good reason.
Add a flag to skip initialization of ImBuf pixels (IB_uninitialized_pixels)
and use that in some parts of VSE effects/rendering/cache/scopes, as well
as image loading code.
Rendering out VSE movie, on Windows/VS2022/Ryzen5950X:
- Sprite Fright: 443sec -> 414sec (takes 93% of previous time)
- Gold previs: 367sec -> 325sec (takes 88% of prev time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118321
Part of "improve filtering situation" (#116980), now strip scaling filter defaults to "Auto" which has logic like:
- No scale, no rotation, integer positions: Nearest (fastest)
- Scaling up by more than 2x: Cubic Mitchell, so you get nicer blending between pixels than with bilinear,
- Scaling down by more than 2x: Box, so that many pixels are averaged properly without too much aliasing,
- Otherwise: Bilinear
Existing strips that use Bilinear (which is default) get switched to Auto when loading older files.
All of this has an advantage that unless you have some special needs for your look, you can leave it at default and it will look decently good at either large up-scaling or large down-scaling, but not waste performance if you don't use any scaling at all. Previously none of the choices were good in "all cases": box (née subsampled3x3) only looks good when scaling down, cubic only looks good when scaling up, default bilinear leaves performance on the table when you don't use any scale/rotation, etc.
On something like Gold movie current edit, most of the strips effectively use Nearest now, except some that are translated into non-integer pixel positions; those stay effectively Bilinear.
Pull Request: https://projects.blender.org/blender/blender/pulls/117853
- "can not" -> "cannot" in many places (ambiguous, also see
Writing Style guide).
- "Bezier" -> "Bézier": proper spelling of the eponym.
- Tool keymaps: make "Uv" all caps.
- "FFMPEG" -> "FFmpeg" (official spelling)
- Use MULTIPLICATION SIGN U+00D7 instead of MULTIPLICATION X U+2715.
- "LClick" -> "LMB", "RClick" -> "RMB": this convention is used
everywhere else.
- "Save rendered the image..." -> "Save the rendered image...": typo.
- "Preserve Current retiming": title case for property.
- Bend status message: punctuation.
- "... class used to define the panel" -> "header": copy-paste error.
- "... class used to define the menu" -> "asset": copy-paste error.
- "Lights user to display objects..." -> "Lights used...": typo.
- "-setaudio require one argument" -> "requires": typo.
Some issues reported by Joan Pujolar and Tamar Mebonia.
Pull Request: https://projects.blender.org/blender/blender/pulls/117856
- enum class StripEarlyOut instead of raw integer defines
- SeqRenderState default initializer instead of seq_render_state_init
- Vector<Sequence*> instead of manually sized arrays of pointers
- some const to several function arguments
Pull Request: https://projects.blender.org/blender/blender/pulls/117829
Part of overall "improve image filtering situation" (#116980), this PR addresses
two issues:
- Bilinear (default) image filtering makes half a source pixel wide transparent
border around the image. This is very noticeable when scaling images/movies up
in VSE. However, when there is no scaling up but you have slightly rotated
image, this creates a "somewhat nice" anti-aliasing around the edge.
- The other filtering kinds (e.g. cubic) do not have this behavior. So they do
not create unexpected transparency when scaling up (yay), however for slightly
rotated images the edge is "jagged" (oh no).
More detail and images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117717
This was caused by global variable `sound_cfra` not being updated when
rendering sequencer data. This global variable could cause problems in
other cases though, so it is removed. Functions that are used to set
anomation buffers now accept frame as argument.
Pull Request: https://projects.blender.org/blender/blender/pulls/117345
Part of overall "improve filtering situation" (#116980): replace Subsampled3x3
(added for blender 3.5 in f210842a72 et al.) strip scaling filter with a
general Box filter.
Subsampled3x3 is really a Box filter ("average pixel values over NxM region"),
hardcoded to 3x3 size. As such, it works pretty well when downscaling images by
3x on each axis. But starts to break down and introduce aliasing at other
scaling factors. Also when scaling up or scaling down by less than 3x, using
total of 9 samples is a bit of overkill and hurts performance.
So instead, calculate the amount of NxM samples needed by looking at scaling
factors on X/Y axes. Note: use at least 2 samples on each axis, so that when
rotation is present, the result edges will get some anti-aliasing, just like it
was happening in previous filter implementation.
Images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117584
Part of overall "improve filtering situation" (#116980) task:
Add "Cubic Mitchell" filtering option to VSE strips. This is a cubic (4x4)
filter that generally looks better than bilinear, while not blurring the image
as much as the Cubic BSpline filter that exists elsewhere within Blender. It is
also default in many other apps.
Rename the (very recently added) VSE Bicubic filter option to Cubic BSpline.
Images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117517
There exist a bunch of "give me a (filtered) image pixel at this location"
functions, some with duplicated functionality, some with almost the same but
not quite, some that look similar but behave slightly differently, etc.
Some of them were in BLI, some were in ImBuf.
This commit tries to improve the situation by:
* Adding low level interpolation functions to `BLI_math_interp.hh`
- With documentation on their behavior,
- And with more unit tests.
* At `ImBuf` level, there are only convenience inline wrappers to the above BLI
functions (split off into a separate header `IMB_interp.hh`). However, since
these wrappers are inline, some things get a tiny bit faster as a side
effect. E.g. VSE image strip, scaling to 4K resolution (Windows/Ryzen5950X):
- Nearest filter: 2.33 -> 1.94ms
- Bilinear filter: 5.83 -> 5.69ms
- Subsampled3x3 filter: 28.6 -> 22.4ms
Details on the functions:
- All of them have `_byte` and `_fl` suffixes.
- They exist in 4-channel byte (uchar4) and float (float4), as well as
explicitly passed amount of channels for other float images.
- New functions in BLI `blender::math` namespace:
- `interpolate_nearest`
- `interpolate_bilinear`
- `interpolate_bilinear_wrap`. Note that unlike previous "wrap" function,
this one no longer requires the caller to do their own wrapping.
- `interpolate_cubic_bspline`. Previous similar function was called just
"bicubic" which could mean many different things.
- Same functions exist in `IMB_interp.hh`, they are just convenience that takes
ImBuf and uses data pointer, width, height from that.
Other bits:
- Renamed `mod_f_positive` to `floored_fmod` (better matches `safe_floored_modf`
and `floored_modulo` that exist elsewhere), made it branchless and added more
unit tests.
- `interpolate_bilinear_wrap_fl` no longer clamps result to 0..1 range. Instead,
moved the clamp to be outside of the call in `paint_image_proj.cc` and
`paint_utils.cc`. Though the need for clamping in there is also questionable.
Pull Request: https://projects.blender.org/blender/blender/pulls/117387
Rename: anim -> ImBufAnim
Rename: anim_index -> ImBufAnimIndex
There were cases where removing redundant "struct" qualifier caused
a warning since the name of the struct member was also anim.
Use uppercase type name to conform with other types names.
Ref !117394
`UUID` generally stands for "universally unique identifier". The session identifier that
we use is neither universally unique, nor does it follow the standard. Therefor, the term
"session uuid" is confusing and should be replaced.
In #116888 we briefly talked about a better name and ended up with "session uid".
The reason for "uid" instead of "id" is that the latter is a very overloaded term in Blender
already.
This patch changes all uses of "uuid" to "uid" where it's used in the context of a
"session uid". It's not always trivial to see whether a specific mention of "uuid" refers
to an actual uuid or something else. Therefore, I might have missed some renames.
I can't think of an automated way to differentiate the case.
BMesh also uses the term "uuid" sometimes in a the wrong context (e.g. `UUIDFaceStepItem`)
but there it also does not mean "session uid", so it's *not* changed by this patch.
Pull Request: https://projects.blender.org/blender/blender/pulls/117350
If a scene sequencer strip references a scene with no composite output,
it will crash due to null image buffer output. This patch fixes that by
allocating an empty buffer in those cases.
Pull Request: https://projects.blender.org/blender/blender/pulls/117135
Before we always did a float conversion of the colors even if we
wouldn't use the result.
Because the float conversion does alpha premultiplication, we can
use some shortcuts to skip a lot of computations
In a edit with a lot of fully transparent pixels, I observed a reduction
in total render time from 1m 31s to 1m 17s.
Pull Request: https://projects.blender.org/blender/blender/pulls/117134
Part of overall "improve filtering situation" (#116980) task:
* Add Bicubic filtering option to strip Transform "Filter" setting.
Previously this option only existed in Transform Effect "Interpolation"
setting.
- With this addition, it feels like the transform effect could
possibly be marked as legacy/deprecated, since the regular Transform
that is on all strips can do everything that Transform Effect did?
* Speed up bicubic filtering (used now in VSE, but also in CPU Compositor,
image paint, etc.) by slightly simplifying the code and using some SIMD.
Upscaling 96x54 image to 3840x2160 resolution, using Bicubic filtering:
- Windows (VS2022, Ryzen 5950X): 35.5ms -> 15.1ms
- Mac (clang 15, M1 Max): 29.6ms -> 24.4ms
* Add gtest coverage for bicubic functionality.
Pull Request: https://projects.blender.org/blender/blender/pulls/117100
Transition retiming keys move as if they are mirrored across a point.
It is possible to allow them to cross this point instead of limiting
transition duration.
This change itself doesn't really improve usability, but it is needed
for "Add transition and change its size" operator.
It's pretty simple, but threading it, and making it write out whole
pixel at a time (instead of one byte at a time) still makes it faster.
4K resolution, five Color strips blended over each other, playback on
Windows/VS2022, Ryzen 5950X:
- Playback 9.2FPS -> 11.5FPS
- do_solid_color for one effect, median time 7.7ms -> 3.8ms
Additionally, the solid color on byte output was not doing float->byte
color rounding & clamping properly, and on float output it was writing
255.0 into alpha instead of 1.0. So fix that too.
Pull Request: https://projects.blender.org/blender/blender/pulls/117058
- Improve the look of them, so they feel less like from year
1998 (more details and images in the PR).
- Some of the scopes got slightly faster in the process, others
stayed the same performance (details below).
- Remove VSE Scopes related data from SpaceSeq DNA, move it into
runtime instead.
This is only visible for very low resolution image strips; the math was
operating on integers as image size but doing division by two to get the
outline. For non-even image sizes the outline could be off by a pixel due
to rounding.
Images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/116605
This also fixes crash when deleting keys.
Issue was caused by incorrect implementation of batch deleting with
`SEQ_retiming_remove_multiple_keys()` function. It tried to remove data
from different strips, when it should work with one strip at the time.
Also transitions and freeze frames were treated as normal keys, but they
do need special handling.
Pull Request: https://projects.blender.org/blender/blender/pulls/116722
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
When slipping meta strip, effect position was not updated. These updates
are limited to VSE core code, so function `SEQ_time_slip_strip()` was
added to resolve this issue.
This also simplifies operator code, so it does not have to deal with
recursion and doesn't need to hold quite detailed strip state for when
operator is cancelled.
Pull Request: https://projects.blender.org/blender/blender/pulls/113200
