Part of overall "improve image filtering situation" (#116980), this PR addresses
two issues:
- Bilinear (default) image filtering makes half a source pixel wide transparent
border around the image. This is very noticeable when scaling images/movies up
in VSE. However, when there is no scaling up but you have slightly rotated
image, this creates a "somewhat nice" anti-aliasing around the edge.
- The other filtering kinds (e.g. cubic) do not have this behavior. So they do
not create unexpected transparency when scaling up (yay), however for slightly
rotated images the edge is "jagged" (oh no).
More detail and images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117717
This was caused by global variable `sound_cfra` not being updated when
rendering sequencer data. This global variable could cause problems in
other cases though, so it is removed. Functions that are used to set
anomation buffers now accept frame as argument.
Pull Request: https://projects.blender.org/blender/blender/pulls/117345
Part of overall "improve filtering situation" (#116980): replace Subsampled3x3
(added for blender 3.5 in f210842a72 et al.) strip scaling filter with a
general Box filter.
Subsampled3x3 is really a Box filter ("average pixel values over NxM region"),
hardcoded to 3x3 size. As such, it works pretty well when downscaling images by
3x on each axis. But starts to break down and introduce aliasing at other
scaling factors. Also when scaling up or scaling down by less than 3x, using
total of 9 samples is a bit of overkill and hurts performance.
So instead, calculate the amount of NxM samples needed by looking at scaling
factors on X/Y axes. Note: use at least 2 samples on each axis, so that when
rotation is present, the result edges will get some anti-aliasing, just like it
was happening in previous filter implementation.
Images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117584
Part of overall "improve filtering situation" (#116980) task:
Add "Cubic Mitchell" filtering option to VSE strips. This is a cubic (4x4)
filter that generally looks better than bilinear, while not blurring the image
as much as the Cubic BSpline filter that exists elsewhere within Blender. It is
also default in many other apps.
Rename the (very recently added) VSE Bicubic filter option to Cubic BSpline.
Images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117517
There exist a bunch of "give me a (filtered) image pixel at this location"
functions, some with duplicated functionality, some with almost the same but
not quite, some that look similar but behave slightly differently, etc.
Some of them were in BLI, some were in ImBuf.
This commit tries to improve the situation by:
* Adding low level interpolation functions to `BLI_math_interp.hh`
- With documentation on their behavior,
- And with more unit tests.
* At `ImBuf` level, there are only convenience inline wrappers to the above BLI
functions (split off into a separate header `IMB_interp.hh`). However, since
these wrappers are inline, some things get a tiny bit faster as a side
effect. E.g. VSE image strip, scaling to 4K resolution (Windows/Ryzen5950X):
- Nearest filter: 2.33 -> 1.94ms
- Bilinear filter: 5.83 -> 5.69ms
- Subsampled3x3 filter: 28.6 -> 22.4ms
Details on the functions:
- All of them have `_byte` and `_fl` suffixes.
- They exist in 4-channel byte (uchar4) and float (float4), as well as
explicitly passed amount of channels for other float images.
- New functions in BLI `blender::math` namespace:
- `interpolate_nearest`
- `interpolate_bilinear`
- `interpolate_bilinear_wrap`. Note that unlike previous "wrap" function,
this one no longer requires the caller to do their own wrapping.
- `interpolate_cubic_bspline`. Previous similar function was called just
"bicubic" which could mean many different things.
- Same functions exist in `IMB_interp.hh`, they are just convenience that takes
ImBuf and uses data pointer, width, height from that.
Other bits:
- Renamed `mod_f_positive` to `floored_fmod` (better matches `safe_floored_modf`
and `floored_modulo` that exist elsewhere), made it branchless and added more
unit tests.
- `interpolate_bilinear_wrap_fl` no longer clamps result to 0..1 range. Instead,
moved the clamp to be outside of the call in `paint_image_proj.cc` and
`paint_utils.cc`. Though the need for clamping in there is also questionable.
Pull Request: https://projects.blender.org/blender/blender/pulls/117387
Rename: anim -> ImBufAnim
Rename: anim_index -> ImBufAnimIndex
There were cases where removing redundant "struct" qualifier caused
a warning since the name of the struct member was also anim.
Use uppercase type name to conform with other types names.
Ref !117394
`UUID` generally stands for "universally unique identifier". The session identifier that
we use is neither universally unique, nor does it follow the standard. Therefor, the term
"session uuid" is confusing and should be replaced.
In #116888 we briefly talked about a better name and ended up with "session uid".
The reason for "uid" instead of "id" is that the latter is a very overloaded term in Blender
already.
This patch changes all uses of "uuid" to "uid" where it's used in the context of a
"session uid". It's not always trivial to see whether a specific mention of "uuid" refers
to an actual uuid or something else. Therefore, I might have missed some renames.
I can't think of an automated way to differentiate the case.
BMesh also uses the term "uuid" sometimes in a the wrong context (e.g. `UUIDFaceStepItem`)
but there it also does not mean "session uid", so it's *not* changed by this patch.
Pull Request: https://projects.blender.org/blender/blender/pulls/117350
If a scene sequencer strip references a scene with no composite output,
it will crash due to null image buffer output. This patch fixes that by
allocating an empty buffer in those cases.
Pull Request: https://projects.blender.org/blender/blender/pulls/117135
Before we always did a float conversion of the colors even if we
wouldn't use the result.
Because the float conversion does alpha premultiplication, we can
use some shortcuts to skip a lot of computations
In a edit with a lot of fully transparent pixels, I observed a reduction
in total render time from 1m 31s to 1m 17s.
Pull Request: https://projects.blender.org/blender/blender/pulls/117134
Part of overall "improve filtering situation" (#116980) task:
* Add Bicubic filtering option to strip Transform "Filter" setting.
Previously this option only existed in Transform Effect "Interpolation"
setting.
- With this addition, it feels like the transform effect could
possibly be marked as legacy/deprecated, since the regular Transform
that is on all strips can do everything that Transform Effect did?
* Speed up bicubic filtering (used now in VSE, but also in CPU Compositor,
image paint, etc.) by slightly simplifying the code and using some SIMD.
Upscaling 96x54 image to 3840x2160 resolution, using Bicubic filtering:
- Windows (VS2022, Ryzen 5950X): 35.5ms -> 15.1ms
- Mac (clang 15, M1 Max): 29.6ms -> 24.4ms
* Add gtest coverage for bicubic functionality.
Pull Request: https://projects.blender.org/blender/blender/pulls/117100
Transition retiming keys move as if they are mirrored across a point.
It is possible to allow them to cross this point instead of limiting
transition duration.
This change itself doesn't really improve usability, but it is needed
for "Add transition and change its size" operator.
It's pretty simple, but threading it, and making it write out whole
pixel at a time (instead of one byte at a time) still makes it faster.
4K resolution, five Color strips blended over each other, playback on
Windows/VS2022, Ryzen 5950X:
- Playback 9.2FPS -> 11.5FPS
- do_solid_color for one effect, median time 7.7ms -> 3.8ms
Additionally, the solid color on byte output was not doing float->byte
color rounding & clamping properly, and on float output it was writing
255.0 into alpha instead of 1.0. So fix that too.
Pull Request: https://projects.blender.org/blender/blender/pulls/117058
This is only visible for very low resolution image strips; the math was
operating on integers as image size but doing division by two to get the
outline. For non-even image sizes the outline could be off by a pixel due
to rounding.
Images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/116605
This also fixes crash when deleting keys.
Issue was caused by incorrect implementation of batch deleting with
`SEQ_retiming_remove_multiple_keys()` function. It tried to remove data
from different strips, when it should work with one strip at the time.
Also transitions and freeze frames were treated as normal keys, but they
do need special handling.
Pull Request: https://projects.blender.org/blender/blender/pulls/116722
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
When slipping meta strip, effect position was not updated. These updates
are limited to VSE core code, so function `SEQ_time_slip_strip()` was
added to resolve this issue.
This also simplifies operator code, so it does not have to deal with
recursion and doesn't need to hold quite detailed strip state for when
operator is cancelled.
Pull Request: https://projects.blender.org/blender/blender/pulls/113200
This was caused by 5c76c7bf84, which only updated sound when tagged by
`ID_RECALC_AUDIO`. On undo or rendering, this flag is missing.
Update equalizer when recalc is flagged by `ID_RECALC_COPY_ON_WRITE`.
Pull Request: https://projects.blender.org/blender/blender/pulls/116282
Split the code, use preconditions, use rather plain language for
function names and add comments where it is not totally obvious, what
the code is supposed to do.
The main problem is, that retiming key frame index is defined in strip
content space (0 <> seq->len -1), but API functions must rescale this
index by applying `SEQ_time_media_playback_rate_factor_get()` and return
value in timeline space.
This wasn't done properly, in many places, some had challenges:
- `SEQ_retiming_key_timeline_frame_get()` returned floats, but UI
expects integers. Otherwise keys may be drawn inbetween frames.
- Function `right_fake_key_frame_get()` must return exact frame of
keys, otherwise lookup by frame would fail. But `retime_key_draw()`
can not compensate position of last fake key, so this has to be done
in `fake_keys_draw()` and `try_to_realize_virtual_key()`.
- For transformation to work as expected, double precision value has
to be used for frame index.
- For UI either API would had to be extended to provide helper functions
to deal with FPS mismatch, or it needs to know the FPS difference.
I have opted to put `SEQ_time_media_playback_rate_factor_get()` in
"public" headers. Neither solution is great.
Now that the code is in C++, quite some duplication between "byte" and
"float" effect code paths can be reduced (easier than it was in C times).
So I did that, removing about 400 lines of code.
In that process I accidentally made Gaussian Blur faster, since while
reducing the amount of code I noticed it was doing some things
sub-optimally (calculated kernel tables for each job, etc.). Applying
100x100 gaussian blur on 4K UHD resolution image strip on Ryzen 5950X
went 630ms -> 450ms.
Pull Request: https://projects.blender.org/blender/blender/pulls/116089
Add "Preserve Current Retiming" option to set speed operator. This
option is enabled by default. When changing speed of retiming segment,
the strip changes length instead of changing next segment speed.
Ref: #112343
Content cut off by `anim_startofs` is as if it does not exist for
sequencer. But Audaspace seeking relies on having animation buffer
initialized for whole sequence. Initialize pitch of cut content to 1.
This is a smaller rewrite/refactor of the VSE copy paste code to use the file copy buffer logic that is used in other places of Blender.
This makes Blender able to copy paste between Blender processes.
It can also paste successfully after closing and reopening Blender.
Other than that, the functionally should remain the exact same as the current copy paste operator with one exception: Scene strips does not retain their scene pointer when pasting into the same file.
This is to make it consistent with how it was before when copy pasting between .blend files.
(The scene data for the scene strips were never copied in when doing that)
Logic for pulling in or linking scenes into new or current files are purposely left for later when a proper proposal of how this would work in a nice fashion is done.
Now the strip data gets copied either fully or partially besides for scene strips. There only the script data gets copied and not the scene data.
If there is a audio/video data block of the same name as in the paste data, it will be reused to reduce potential duplication of data.
Pull Request: https://projects.blender.org/blender/blender/pulls/114703
Wipe effect was completely single threaded. So multi-thread that.
Additionally, simplify math done in Clock wipe code; asin() + hypot()
+ some checks is just atan2() really.
Applying Wipe to 4K UHD sequencer output, on Windows Ryzen 5950X:
- Single/Double: 99.1 -> 9.3 ms (10.6x faster)
- Iris: 153.3 -> 12.3 ms (12.4x faster)
- Clock: 301.4 -> 14.5 ms (20.8x faster)
The same on Mac M1 Max:
- Single: 74.5 -> 13.4 ms (5.6x faster)
- Iris: 84.2 -> 14.3 ms (5.9x faster)
- Clock: 185.3 -> 18.8 ms (9.8x faster)
Pull Request: https://projects.blender.org/blender/blender/pulls/115837
Glow effect was doing the correct thing algorithmically (separable gaussian
blur), but it was 1) completely single-threaded, and 2) did operations in
several passes over the source images, instead of doing them in one go.
- Adds multi-threading to Glow effect.
- Combines some operations, e.g. instead of IMB_buffer_float_from_byte
followed by IMB_buffer_float_premultiply, do
IMB_colormanagement_transform_from_byte_threaded which achieves the same,
but more efficiently.
- Simplifies the code: removing separate loops around image boundaries is
both less code and slightly faster; use float4 vector type for more
compact code; use Array classes instead of manual memory allocation, etc.
- Removes IMB_buffer_float_unpremultiply and IMB_buffer_float_premultiply
since they are no longer used by anything whatsoever.
Applying Glow to 4K UHD sequencer output, on Windows Ryzen 5950X:
- Blur distance 4: 935ms -> 109ms (8.5x faster)
- Blur distance 20: 3526ms -> 336ms (10.5x faster)
Same on Mac M1 Max:
- Blur distance 4: 732ms -> 126ms (5.8x faster)
- Blur distance 20: 3047ms -> 528ms (5.7x faster)
Pull Request: https://projects.blender.org/blender/blender/pulls/115818
Gamma Cross code seems to be coming from year 2005 or earlier, with complex
table based machinery to approximate "raise to power" calculations. Which,
for Gamma Cross, have always been hardcoded to 2.0 "since forever". So
simplify all that, i.e. replace all the table lookup/interpolation things
with just `x*x` and `sqrt(x)`.
Applying Gamma Cross on 4K UHD resolution, Windows Ryzen 5950X machine:
36.2ms -> 8.1ms
Pull Request: https://projects.blender.org/blender/blender/pulls/115801
Sequencer timeline UI repainting is 3x-4x faster now, for complex
timelines. On Sprite Fright Edit data set, with whole timeline visible
(2702 strips), repainting the timeline UI with all overlay options
(waveforms, offsets, thumbnails etc.):
- Windows (Ryzen 5950X, RTX 3080Ti, OpenGL): 62ms -> 18.6ms (16FPS -> 54FPS)
- Mac (M1 Max, Metal): 39.8ms -> 11.5ms (25FPS -> 86FPS)
This is achieved by:
- Avoiding tiny GPU draw calls (i.e. drawing one quad a time), instead
batch all the quads / lines needed by the timeline display into
series of about-1000 quads per draw.
- For retiming keys display, batch their keyframe point drawing too.
- For audio waveform overlay display, change it to draw batched quads
instead of alternating between line strips and triangle strips. This
actually changes how the waveform looks like (implements #115274)
and fixes some visual issues with waveforms too.
- For fcurve overlays, also draw them as batched quads.
While at it, this also fixes an issue where while dragging strips over
other strips, their text labels would look as if they are behind the
background strips.
Pull Request: https://projects.blender.org/blender/blender/pulls/115311
Since b1526dd2c6, viewport renderer sets `G.is_rendering`, but VSE scene
rendering function used this to decide whether to do offscreen opengl
render or use render API. This logic was quite weak.
Use `SeqRenderData::for_render` instead of `G.is_rendering`, since it
explicitly defines whether strip rendering is invoked by F12 render job.
Since offscreen gl rendering rely on `BLI_thread_is_main()` being
true, use this to initialize `do_seq_gl` variable for clarity.
The use of render job path was further conditioned on `G.is_rendering`,
with exception of running headless, but this condition was incorrect.
This condition was reformulated as precondition, which if true, returns
`nullptr` instead of crashing.
Pull Request: https://projects.blender.org/blender/blender/pulls/115079
