Speedup the Color Balance VSE strip modifier, with two things:
- Generally, use a much lower overhead parallel_for, also with
lower grain size (32 image rows, instead of 64 that were used
before). This is what makes the "float" variant faster.
- For "byte" variant, create a precalculated lookup table instead
of doing all the math per-pixel. This was *almost* done in
existing code, except it was put into the code path that was
never-ever used. However, since this is all done on premultiplied
values, I'm using lookup table size of 1024 instead of 256, so
that semitransparent pixels get some more precision for
"in-between values". This LUT is what results in the main speedup
of "byte" variant.
Calculating Color Balance at 4K resolution, times in milliseconds:
- PC (Ryzen 5950X), PNG (byte): 22.2 -> 2.9 ms, EXR (float): 20.1 -> 15.2 ms
- Mac (M1 Max), PNG (byte): 28.9 -> 7.5 ms, EXR (float): 21.8 -> 8.5 ms
More timing details in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127121
VSE timeline, when many (hundreds/thousands) of thumbnails were visible, was
very slow to redraw. This PR makes them 3-10x faster to redraw, by stopping
doing things that are slow :) Part of #126087 thumbnail improvements task.
- No longer do mute semitransparency or corner rounding on the CPU, do it in
shader instead.
- Stop creating a separate GPU texture for each thumbnail, on every repaint,
and drawing each thumbnail as a separate draw call. Instead, put thumbnails
into a single texture atlas (using a simple shelf packing algorithm), and
draw them in batch, passing data via UBO. The atlas is still re-created every
frame, but that does not seem to be a performance issue. Thumbnails are
cropped horizontally based on how much of their parts are visible (e.g. a
narrow strip on screen), so realistically the atlas size is kinda
proportional to screen size, and ends up being just several megabytes of data
transfer between CPU -> GPU each frame.
On this Sprite Fright edit timeline view (612 visible thumbnails), time taken
to repaint the timeline window:
- Mac (M1 Max, Metal): 68.1ms -> 4.7ms
- Windows (Ryzen 5950X, RTX 3080Ti, OpenGL): 23.7ms -> 6.8ms
This also fixes a visual issue with thumbnails, where when strips are very
tall, the "rounded corners" that were poked right into the thumbnail bitmap
on the CPU were showing up due to actual bitmap being scaled up a lot.
Pull Request: https://projects.blender.org/blender/blender/pulls/126972
Implementing part of design outlined in #126087.
- VSE thumbnail cache has a new implementation, hopefully simpler
and easier to understand.
- Instead of cache key being a VSE strip, frame index, plus
complicated logic for cache items linking etc.,
- The cache is keyed by media file path (if multiple strips
use the same input file, they will share cache entries), frame
index within media file, and any extra data (e.g. steam index
for multi-steam videos)
- Much reduced cache flickering and strange/weird thumbnail choices.
- Likewise, thumbnails no longer disappear-and-reload on operations
like Undo, dragging new video strip into timeline, or F12 render.
- Thumbnails now load faster.
- Images use dedicated/faster thumbnail loading routines when a
format can do that (e.g. JPG and EXR can).
- Movies reuse ffmpeg decoding context for neighboring strips
that use the same file (as often happens when cutting footage)
- Thumbnail requests are processed on several threads now too.
Images and more detail in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126405
Adds the ability to connect and disconnect strips in the VSE.
- Connected strips have an icon indicating their status, and attempting
to select one connected strip selects all other connected strips in
that chain.
- If the user attempts to connect a strip that is already connected to
other strips, that strip will disconnect itself from others before
connecting to new strips.
- Preview selection also works in bulk if multiple video strips are
connected together in the timeline.
- When adding new strips from the Add menu or the File Browser, strips
from the same file are connected by default. There's an option to
turn this off in Editing > Video Sequencer user preferences.
- It is possible to individually tweak strips/handles and ignore
connections with Alt+Click.
- This shortcut overrides the old keymap item for "Linked Handle"
selection. The property still exists if people want to use that
shortcut for its old purpose.
- To make sure that connections remain valid even after duplication,
I've added a condition to `seq_new_fix_links_recursive` that also
updates connections using the `seq->tmp` var. (A note -- I've updated
the comment for this field in `DNA_sequence_types.h` because the var
is only used for duplication now. It was once present in
`select_more_less_seq__internal` to be used for linked selection but
is gone now).
- There are also functions to cut one-way links and make sure that
all strips are bidirectionally connected after duplicating.
Pull Request: https://projects.blender.org/blender/blender/pulls/124333
API: merged IMB_scalefastImBuf, IMB_scaleImBuf, IMB_scaleImBuf_threaded
into one function IMB_scale with enum IMBScaleFilter {Nearest, Bilinear, Box}
and bool "threaded" param.
Performance:
- Box filtering (nee IMB_scaleImBuf) can be multi-threaded now.
- Nearest filtering (nee IMB_scalefastImBuf) can be multi-threaded now.
Also fix performance regression on float images caused by fix in #126234
- Bilinear filtering (nee IMB_scaleImBuf_threaded) is several times faster now.
Correctness:
- Nearest and Box filtering: no longer loses half of edge pixels when scaling
up.
- Box: fixed garbage results (and possible out of bounds reads) for non-4
channel float images.
- Bilinear: no longer shifts image when scaling up.
- Bilinear: properly filters when scaling down by 2x2.
Test coverage:
- Add gtest coverage for various IMB_scale modes.
- Add a IMB_performance_test performance test, ran manually.
More details, images and performance numbers in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126390
SEQ_transform_sequence_can_be_translated, SEQ_transform_single_image_check,
SEQ_can_use_proxy can all trivially take const Sequence argument.
Made SEQ_give_frame_index take const Sequence too, by removing the
"modify seq strobe to be at least 1.0" code. The strobe itself is only
ever used inside the same function, and is already guarded by
"is strobe > 1.0" check.
Original code seems to be coming all the way from 2009 with
commit message "2.5. 12k lines of sequencer back!" (03fc5696dc).
Pull Request: https://projects.blender.org/blender/blender/pulls/126021
Prevent VSE cache returning image when running render job.
Previously, before render job was started, cache was cleared, but since
disk cache is not cleared, it returned an image. Now cache is not
cleared, but it does not return or store images when rendering.
Pull Request: https://projects.blender.org/blender/blender/pulls/125947
Match function and declaration names, picking names based on
consistency with related code & clarity.
Also changes for old conventions, missed in previous cleanups:
- name -> filepath
- tname -> newname
- maxlen -> maxncpy
As well as takes text color transparency into account. That is, text
"shadow" now virtually casts shadow of however the text itself + outline
looks like. The result is blurred, and extra shadow color tint and
transparency is applied.
Performance testing: seems to be slightly faster than before in my tests.
Probably because the shadow "image" that is now blurred only contains
one byte per pixel (the transparency), instead of full uchar4 per pixel
(with only the "transparency" being blurred).
I have extended sequencer render tests to better cover various text
outline/shadow/box combinations.
Pull Request: https://projects.blender.org/blender/blender/pulls/125445
This is only visible when text color has transparency or is fully
transparent. Desired behavior is that the outline would only be
"outside" the actual text.
Pull Request: https://projects.blender.org/blender/blender/pulls/125372
The cache invalidation worked on basis of timeline frame instead of
frame index, which caused, that these images were outside of range, that
was considered to be changed, after strip was moved.
In this case the issue happened only with effect strips. This was not
an issue before, mainly because raw images for these strips were not
stored. For other strip types, raw image does not usually change.
This seems to be "caused" by f4f708a54f .
Pull Request: https://projects.blender.org/blender/blender/pulls/124870
If a strip had effect strips applied on it, and using the reset
retiming operator caused the strips to increase in length, the effect
strips would become overlapped with any strips they hit, unless the
"Expand" overlap mode was used.
This is because the "effect chain" strips weren't included
in #strips.
Co-authored-by: Richard Antalik <richardantalik@gmail.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/124750
This commit moves generated `RNA_blender.h`, `RNA_prototype.h` and
`RNA_blender_cpp.h` headers to become C++ header files.
It also removes the now useless `RNA_EXTERN_C` defines, and just
directly use the `extern` keyword. We do not need anymore `extern "C"`
declarations here.
Pull Request: https://projects.blender.org/blender/blender/pulls/124469
Updates the slide operator to be able to do subframe adjustments of
audio strips. If you hold shift while sliding (the S shortcut), it will
start to do subframe adjustments. It is also possible to type in
fractional inputs while in modal mode. (IE start sliding then type 1.5)
This change simplifies access to this state by prefetching code.
Previously it had to iterate over screens from `bMain` in code local to
prefetching. Now this state is available to other code in VSE.
When keys are realized, strip handle positions are used to ensure, that
left and right key exists. If strip content is offset to the right,
this caused crash. This happened, because `SEQ_retiming_add_key()`
clamped timeline frame to strip content only on right side.
Clamp timeline frame to strip content on both sides.
Pull Request: https://projects.blender.org/blender/blender/pulls/124207
Adds snapping in the VSE preview area.
Source points are the four corners and origins of all selected, visible
image quads. Targets can be preview borders, preview center, or
corners/origins of other strips.
Pull Request: https://projects.blender.org/blender/blender/pulls/122759
Code that implemented "show missing media in magenta" in VSE preview
only implemented that for images and movies, and forgot that scene
strips can also be missing.
Pull Request: https://projects.blender.org/blender/blender/pulls/124124
Caused by incorrect output of `SEQ_give_frame_index` for effect strips.
Also since output value is float, it has to be truncated to integer,
when image bufferes are rendered for the effect.
Pull Request: https://projects.blender.org/blender/blender/pulls/123914
When preview is downscaled and transformation origin is not the center
of the image, this causes unexpected offset. This happened, because one
matrix combined image downscaling, so it fits into preview and user
defined scale. When origin was not center of the image, this results in
incorrect offset.
Solved by splitting 1 matrix in `sequencer_image_crop_transform_matrix`
into 2 matrices. First matrix just centers and scales image to expected
size. Second matrix performs rest of transform operations. This code is
bit easier to read as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/123776
The issue was caused by the ImBuf of the scene strip render sharing the float
buffer pixels with the ImBuf from the render result. If the render result is
ever gets freed (i.e., by a request to perform another render) it'll leave the
strip ImBuf pointing to a freed memory.
This was caused by the #109788.
The simple solution is to restore the code to the state prior to the ImBuf
refactor in the RenderRsult. A better solution would be to use implicit
sharing, similar to how it was done in the #108045.
Pull Request: https://projects.blender.org/blender/blender/pulls/123731
