This patch fixes broken connections and effect strips which would arise
in some cases when trying to split connected strips with effects.
This happened in case an effect strip was being split -- first, its
effect chain would be added to a VectorSet of seqs, but then, only the
connected strips of the original seq (in this case the effect strip)
would be added to the VectorSet. Fix by iterating over the entire list
and adding connected strips.
Additionally:
- Add a final SEQ_iterator_set_expand() for SEQ_query_effect_chain() in
case these connected strips had effects themselves.
- Move the seq_edit_split_operation_permitted() check after the entire
chain is evaluated to avoid bugs w/ locked strips being editable
before.
Pull Request: https://projects.blender.org/blender/blender/pulls/128670
Building proxies for images was always saving them as JPG files.
However when input images are floating point (e.g. EXR files), this
loses both range and precision, making the resulting proxies
not really be useful for anything where you'd be using EXR files.
Change this to save float image proxies as EXR instead (FP16 data,
lossy DWAA compression). In my quick tests this does result in about
3x-4x larger proxy file size compared to JPG, however at 50% DWAA
quality that is still 10-30x smaller than original EXRs would be.
Changed proxy image loading to explicitly tell to load metadata. JPGs
were always loading it anyway, but EXRs only load when instructed to
do so.
Pull Request: https://projects.blender.org/blender/blender/pulls/128835
Tmp field was still used in strip duplication. This was a source of bugs
previously. Now it uses `Map` of new to old strips to fix strip
relations. The field is removed from DNA struct.
Pull Request: https://projects.blender.org/blender/blender/pulls/128402
IMB_scale modifies the input image. But some places in code needed to keep
original input intact, so what they did was a sequence of IMB_dupImBuf+IMB_scale
Add IMB_scale_into_new function and use that in several obvious places:
- movieclip_build_proxy_ibuf
- icon_copy_rect
- seq_proxy_build_frame
Rebuilding proxies for VSE image sequences with 94 4K resolution EXR images
(on Ryzen 5950X/Win10/VS2022): 13.4 -> 10.3 seconds.
Pull Request: https://projects.blender.org/blender/blender/pulls/128752
This is a follow-up to #128363, and fixes up the remaining areas of
the sequencer's code that didn't yet account for slotted actions:
1. Moving strips failed to move their animation with them.
2. Duplicating strips failed to duplicate their animation with them.
3. Deleting strips didn't delete their animation channels with them.
This also takes the opportunity to add depsgraph tagging and
notifiers that were already missing in the pre-slotted-actions
code, for the strip delete and strip paste operators. The absence
of these was making the UI not update and was also causing stale
animation data to get evaluated.
Pull Request: https://projects.blender.org/blender/blender/pulls/128440
Previously, alignment did exist, but it only changed whole text block
position in relation to a fixed point. This was later renamed to "Anchor".
Now it correctly aligns each line of text. Alignment works with newline
character and word wrapping.
Currently newline characters can't be entered directly, but this should
be resolved soon.
To keep existing anchoring feature, new DNA fields are added and
values from old alignment are copied there.
This PR is part of bigger change [1], and originally I expected to
implement this feature at later stage. But the design called for drawing
text character by character, which would mean, that I would have to
rewrite text alignment anyway.
To render the text, a struct is built, where position and width of each
character is stored. In addition, width of each line is stored. This allows
to implement proper text alignment feature, instead of existing
anchoring. Text is then drawn character by character in a loop.
There are some small differences in text rendering, since this is only
approximation of how BLF library draws glyphs, but it is very close.
For text bounbox, `BLF_boundbox()` is used on per line basis,
because some fonts do not use their full height and this information
is not available on VSE side.
[1] https://projects.blender.org/blender/blender/issues/126547
Pull Request: https://projects.blender.org/blender/blender/pulls/126660
Before this, copy-pasting sequencer strips with animation on them would
fail to copy-paste their animation along with them if they were animated
by a slotted action. This fixes that.
There are three remaining known issues in the sequencer when used with
slotted actions that this PR does not fix, and will be addressed in a
follow-up PR:
1. Moving strips fails to move the animation of their fcurves with them,
which it should.
2. Duplicating strips (as distinct from copying followed by pasting)
fails to duplicate their animation with them.
3. Deleting a strip does not delete its animation channels with it.
Pull Request: https://projects.blender.org/blender/blender/pulls/128363
If conditions are not met for offscreen rendering, render job is used as
fallback. This can be skipped if there is possibility of race condition.
This was only considered for case where `context->for_render` was false,
since this indicates that call is invoked by the job itself.
If scene strip display is set to `OB_RENDER`, it needs to start the
render job. So variable `do_seq_gl` was renamed to `is_preview` and
reused in race avoidance connditon. This was done to convey clearer
meaning and match comments.
This change does make code less safe - race is avoided by UI not
allowing to run F12 render and opengl render at the same time. However,
I don't think, this could be easily resolved on VSE side without
suspending thread or being able to run multiple render jobs.
Pull Request: https://projects.blender.org/blender/blender/pulls/127926
+0.5 should be added to integer pixel coordinate (sampling at pixel
center). This is only really visible in very specific scenarios, but
is more correct.
When adding sound strips, it would in some cases be offset
from the desired start position.
This was because some sound files had a start time that was not 0
even if they didn't have any video file associated with them.
Only try to shift the sound strip around if it is added with a video
strip.
Pull Request: https://projects.blender.org/blender/blender/pulls/127256
When I was learning VSE code, MAXSEQ constant (a preprocessor define!) was
confusing. It makes it sound like it is "max sequences", but it is actually
"max channels". So rename it to SEQ_MAX_CHANNELS and make it C++ constexpr int
instead of preprocessor macro.
Pull Request: https://projects.blender.org/blender/blender/pulls/128024
Drawing the list of channels on the left side of VSE timeline was taking
surprisingly long time (3.5ms in Sprite Fright Edit v135 on Mac M1 Max).
This PR makes them draw in 0.4ms, i.e. 10x faster, by doing two things:
- Stop "search through the whole timeline" work whenever we need to know
which meta strip (if any) owns some VSE channel. Remember that info in
the "sequence lookup" that already holds data to speedup similar queries
(sequence by name, meta by sequence). I.e. this gains "channel owner by
channel" query hashtable.
- Stop alternating between "regular font glyph cache" and "svg icons glyph
cache" on each channel row, which incurs a draw call / UI batch flush.
Instead, draw all the lock/mute widgets first, then all the channel names.
While at it, simplify code related to sequence lookup:
- Use C++ mutex with lock_guard
- Use BLI Map instead of GHash
- Simplify SEQ_sequence_lookup_tag to just SEQ_sequence_lookup_invalidate
- Remove unused SEQ_get_meta_by_seqbase and SEQ_query_all_meta_strips_recursive
Pull Request: https://projects.blender.org/blender/blender/pulls/127913
VSE tonemap is 12-15 times faster.
1) multi-threaded image luminance calculation,
2) avoid calling into OpenColorIO per-pixel, instead do that in
larger batches,
3) for float images (which are primary target for tonemapping),
do not do "to linear space" conversion twice.
Applying tonemap on 4K resolution EXR image, on Ryzen 5950X (Win10/VS2022):
- R/D Photoreceptor mode: 405 -> 31 ms
- Rh Simple mode: 388 -> 23 ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127467
To avoid unnecessary looping over listbase items the function
`BLI_listbase_count_at_most` was used however it resulting in an awkward
expression: `BLI_listbase_count_at_most(list, count + 1) == count`
replace this with `BLI_listbase_count_is_equal_to(list, count)`.
Do not try to kill thumbnail generation job when destroying the cache
(which happens as part of destroying the scene) -- all the code
paths that destroy a scene already cancel outstanding WM jobs. And WM
itself might be gone at that point, so accessing a stale pointer to it
can lead to a crash.
Instead, fix the problem of "refresh sequencer can cause a crash" by
not destroying the thumbnail cache, but merely clearing it in
sequencer_refresh_all_exec.
Pull Request: https://projects.blender.org/blender/blender/pulls/127485
"seq3" input for VSE effect strips has been there ever since
"initial revision" commit in 2002, with comment "pointers voor effecten"
even. But it has never been used, so remove all code that pretends
to do something with it.
Pull Request: https://projects.blender.org/blender/blender/pulls/127401
When doing average/min/max luminance calculations for tonemapping, ignore
pixels that are outside of the strip rectangle. Due to how VSE innards work,
when a strip is positioned to not cover the whole screen, "the rest" is filled
with transparent black. For tonemapping, this was dragging average calculated
luminance way down.
Images of the issue in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127207
Image/movie strip controls color controls: "Saturation" and "Multiply" were
both single-threaded code. Multi-thread both of them. Timings to apply them
on a 4K resolution byte image, on Ryzen 5950X (Win10/VS2022):
- Saturation: 97ms -> 6.3ms
- Multiply: 11.5ms -> 1.1ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127409
SEQ_modifier_apply_stack is called from exactly one place, and it always
was throwing away the input image, and taking the newly produced image.
Which means, we can stop copying it. Just run the modifiers on the
input image.
In a test file that is HD (1080p) resolution, two EXR image sequences,
adjustment layer with Color Balance, and a text strip on top,
playback framerate (Ryzen 5950X, Win10/VS2022) goes 20.8 -> 22.1 fps
Pull Request: https://projects.blender.org/blender/blender/pulls/127346
When a strip with alpha-over blend mode (which is default) has an alpha
channel and is at the bottom of the whole stack, there's no point in
blending it with black color; the result will be identical.
So stop doing that. Also, whenever some other case happens at the bottom
of the stack (e.g. some other blend mode), create the fake "black input"
image at the needed color type (float or byte) and with the correct
color space already applied. Otherwise, especially for floats, VSE
spends much time converting this fake black input from byte to float,
and then converting it to sequencer color space.
Test case of two EXR files blended over each other, at 4K resolution,
on Ryzen 5950X (Win10/VS2022):
- whole sequencer_preview_region_draw 115ms -> 75ms
- seq_render_give_ibuf part: 64ms -> 24ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127310
Speedup the Color Balance VSE strip modifier, with two things:
- Generally, use a much lower overhead parallel_for, also with
lower grain size (32 image rows, instead of 64 that were used
before). This is what makes the "float" variant faster.
- For "byte" variant, create a precalculated lookup table instead
of doing all the math per-pixel. This was *almost* done in
existing code, except it was put into the code path that was
never-ever used. However, since this is all done on premultiplied
values, I'm using lookup table size of 1024 instead of 256, so
that semitransparent pixels get some more precision for
"in-between values". This LUT is what results in the main speedup
of "byte" variant.
Calculating Color Balance at 4K resolution, times in milliseconds:
- PC (Ryzen 5950X), PNG (byte): 22.2 -> 2.9 ms, EXR (float): 20.1 -> 15.2 ms
- Mac (M1 Max), PNG (byte): 28.9 -> 7.5 ms, EXR (float): 21.8 -> 8.5 ms
More timing details in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127121
VSE timeline, when many (hundreds/thousands) of thumbnails were visible, was
very slow to redraw. This PR makes them 3-10x faster to redraw, by stopping
doing things that are slow :) Part of #126087 thumbnail improvements task.
- No longer do mute semitransparency or corner rounding on the CPU, do it in
shader instead.
- Stop creating a separate GPU texture for each thumbnail, on every repaint,
and drawing each thumbnail as a separate draw call. Instead, put thumbnails
into a single texture atlas (using a simple shelf packing algorithm), and
draw them in batch, passing data via UBO. The atlas is still re-created every
frame, but that does not seem to be a performance issue. Thumbnails are
cropped horizontally based on how much of their parts are visible (e.g. a
narrow strip on screen), so realistically the atlas size is kinda
proportional to screen size, and ends up being just several megabytes of data
transfer between CPU -> GPU each frame.
On this Sprite Fright edit timeline view (612 visible thumbnails), time taken
to repaint the timeline window:
- Mac (M1 Max, Metal): 68.1ms -> 4.7ms
- Windows (Ryzen 5950X, RTX 3080Ti, OpenGL): 23.7ms -> 6.8ms
This also fixes a visual issue with thumbnails, where when strips are very
tall, the "rounded corners" that were poked right into the thumbnail bitmap
on the CPU were showing up due to actual bitmap being scaled up a lot.
Pull Request: https://projects.blender.org/blender/blender/pulls/126972
Implementing part of design outlined in #126087.
- VSE thumbnail cache has a new implementation, hopefully simpler
and easier to understand.
- Instead of cache key being a VSE strip, frame index, plus
complicated logic for cache items linking etc.,
- The cache is keyed by media file path (if multiple strips
use the same input file, they will share cache entries), frame
index within media file, and any extra data (e.g. steam index
for multi-steam videos)
- Much reduced cache flickering and strange/weird thumbnail choices.
- Likewise, thumbnails no longer disappear-and-reload on operations
like Undo, dragging new video strip into timeline, or F12 render.
- Thumbnails now load faster.
- Images use dedicated/faster thumbnail loading routines when a
format can do that (e.g. JPG and EXR can).
- Movies reuse ffmpeg decoding context for neighboring strips
that use the same file (as often happens when cutting footage)
- Thumbnail requests are processed on several threads now too.
Images and more detail in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126405
Adds the ability to connect and disconnect strips in the VSE.
- Connected strips have an icon indicating their status, and attempting
to select one connected strip selects all other connected strips in
that chain.
- If the user attempts to connect a strip that is already connected to
other strips, that strip will disconnect itself from others before
connecting to new strips.
- Preview selection also works in bulk if multiple video strips are
connected together in the timeline.
- When adding new strips from the Add menu or the File Browser, strips
from the same file are connected by default. There's an option to
turn this off in Editing > Video Sequencer user preferences.
- It is possible to individually tweak strips/handles and ignore
connections with Alt+Click.
- This shortcut overrides the old keymap item for "Linked Handle"
selection. The property still exists if people want to use that
shortcut for its old purpose.
- To make sure that connections remain valid even after duplication,
I've added a condition to `seq_new_fix_links_recursive` that also
updates connections using the `seq->tmp` var. (A note -- I've updated
the comment for this field in `DNA_sequence_types.h` because the var
is only used for duplication now. It was once present in
`select_more_less_seq__internal` to be used for linked selection but
is gone now).
- There are also functions to cut one-way links and make sure that
all strips are bidirectionally connected after duplicating.
Pull Request: https://projects.blender.org/blender/blender/pulls/124333
API: merged IMB_scalefastImBuf, IMB_scaleImBuf, IMB_scaleImBuf_threaded
into one function IMB_scale with enum IMBScaleFilter {Nearest, Bilinear, Box}
and bool "threaded" param.
Performance:
- Box filtering (nee IMB_scaleImBuf) can be multi-threaded now.
- Nearest filtering (nee IMB_scalefastImBuf) can be multi-threaded now.
Also fix performance regression on float images caused by fix in #126234
- Bilinear filtering (nee IMB_scaleImBuf_threaded) is several times faster now.
Correctness:
- Nearest and Box filtering: no longer loses half of edge pixels when scaling
up.
- Box: fixed garbage results (and possible out of bounds reads) for non-4
channel float images.
- Bilinear: no longer shifts image when scaling up.
- Bilinear: properly filters when scaling down by 2x2.
Test coverage:
- Add gtest coverage for various IMB_scale modes.
- Add a IMB_performance_test performance test, ran manually.
More details, images and performance numbers in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126390
SEQ_transform_sequence_can_be_translated, SEQ_transform_single_image_check,
SEQ_can_use_proxy can all trivially take const Sequence argument.
Made SEQ_give_frame_index take const Sequence too, by removing the
"modify seq strobe to be at least 1.0" code. The strobe itself is only
ever used inside the same function, and is already guarded by
"is strobe > 1.0" check.
Original code seems to be coming all the way from 2009 with
commit message "2.5. 12k lines of sequencer back!" (03fc5696dc).
Pull Request: https://projects.blender.org/blender/blender/pulls/126021
Prevent VSE cache returning image when running render job.
Previously, before render job was started, cache was cleared, but since
disk cache is not cleared, it returned an image. Now cache is not
cleared, but it does not return or store images when rendering.
Pull Request: https://projects.blender.org/blender/blender/pulls/125947
