If conditions are not met for offscreen rendering, render job is used as
fallback. This can be skipped if there is possibility of race condition.
This was only considered for case where `context->for_render` was false,
since this indicates that call is invoked by the job itself.
If scene strip display is set to `OB_RENDER`, it needs to start the
render job. So variable `do_seq_gl` was renamed to `is_preview` and
reused in race avoidance connditon. This was done to convey clearer
meaning and match comments.
This change does make code less safe - race is avoided by UI not
allowing to run F12 render and opengl render at the same time. However,
I don't think, this could be easily resolved on VSE side without
suspending thread or being able to run multiple render jobs.
Pull Request: https://projects.blender.org/blender/blender/pulls/127926
+0.5 should be added to integer pixel coordinate (sampling at pixel
center). This is only really visible in very specific scenarios, but
is more correct.
When adding sound strips, it would in some cases be offset
from the desired start position.
This was because some sound files had a start time that was not 0
even if they didn't have any video file associated with them.
Only try to shift the sound strip around if it is added with a video
strip.
Pull Request: https://projects.blender.org/blender/blender/pulls/127256
When I was learning VSE code, MAXSEQ constant (a preprocessor define!) was
confusing. It makes it sound like it is "max sequences", but it is actually
"max channels". So rename it to SEQ_MAX_CHANNELS and make it C++ constexpr int
instead of preprocessor macro.
Pull Request: https://projects.blender.org/blender/blender/pulls/128024
Drawing the list of channels on the left side of VSE timeline was taking
surprisingly long time (3.5ms in Sprite Fright Edit v135 on Mac M1 Max).
This PR makes them draw in 0.4ms, i.e. 10x faster, by doing two things:
- Stop "search through the whole timeline" work whenever we need to know
which meta strip (if any) owns some VSE channel. Remember that info in
the "sequence lookup" that already holds data to speedup similar queries
(sequence by name, meta by sequence). I.e. this gains "channel owner by
channel" query hashtable.
- Stop alternating between "regular font glyph cache" and "svg icons glyph
cache" on each channel row, which incurs a draw call / UI batch flush.
Instead, draw all the lock/mute widgets first, then all the channel names.
While at it, simplify code related to sequence lookup:
- Use C++ mutex with lock_guard
- Use BLI Map instead of GHash
- Simplify SEQ_sequence_lookup_tag to just SEQ_sequence_lookup_invalidate
- Remove unused SEQ_get_meta_by_seqbase and SEQ_query_all_meta_strips_recursive
Pull Request: https://projects.blender.org/blender/blender/pulls/127913
VSE tonemap is 12-15 times faster.
1) multi-threaded image luminance calculation,
2) avoid calling into OpenColorIO per-pixel, instead do that in
larger batches,
3) for float images (which are primary target for tonemapping),
do not do "to linear space" conversion twice.
Applying tonemap on 4K resolution EXR image, on Ryzen 5950X (Win10/VS2022):
- R/D Photoreceptor mode: 405 -> 31 ms
- Rh Simple mode: 388 -> 23 ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127467
To avoid unnecessary looping over listbase items the function
`BLI_listbase_count_at_most` was used however it resulting in an awkward
expression: `BLI_listbase_count_at_most(list, count + 1) == count`
replace this with `BLI_listbase_count_is_equal_to(list, count)`.
Do not try to kill thumbnail generation job when destroying the cache
(which happens as part of destroying the scene) -- all the code
paths that destroy a scene already cancel outstanding WM jobs. And WM
itself might be gone at that point, so accessing a stale pointer to it
can lead to a crash.
Instead, fix the problem of "refresh sequencer can cause a crash" by
not destroying the thumbnail cache, but merely clearing it in
sequencer_refresh_all_exec.
Pull Request: https://projects.blender.org/blender/blender/pulls/127485
"seq3" input for VSE effect strips has been there ever since
"initial revision" commit in 2002, with comment "pointers voor effecten"
even. But it has never been used, so remove all code that pretends
to do something with it.
Pull Request: https://projects.blender.org/blender/blender/pulls/127401
When doing average/min/max luminance calculations for tonemapping, ignore
pixels that are outside of the strip rectangle. Due to how VSE innards work,
when a strip is positioned to not cover the whole screen, "the rest" is filled
with transparent black. For tonemapping, this was dragging average calculated
luminance way down.
Images of the issue in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127207
Image/movie strip controls color controls: "Saturation" and "Multiply" were
both single-threaded code. Multi-thread both of them. Timings to apply them
on a 4K resolution byte image, on Ryzen 5950X (Win10/VS2022):
- Saturation: 97ms -> 6.3ms
- Multiply: 11.5ms -> 1.1ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127409
SEQ_modifier_apply_stack is called from exactly one place, and it always
was throwing away the input image, and taking the newly produced image.
Which means, we can stop copying it. Just run the modifiers on the
input image.
In a test file that is HD (1080p) resolution, two EXR image sequences,
adjustment layer with Color Balance, and a text strip on top,
playback framerate (Ryzen 5950X, Win10/VS2022) goes 20.8 -> 22.1 fps
Pull Request: https://projects.blender.org/blender/blender/pulls/127346
When a strip with alpha-over blend mode (which is default) has an alpha
channel and is at the bottom of the whole stack, there's no point in
blending it with black color; the result will be identical.
So stop doing that. Also, whenever some other case happens at the bottom
of the stack (e.g. some other blend mode), create the fake "black input"
image at the needed color type (float or byte) and with the correct
color space already applied. Otherwise, especially for floats, VSE
spends much time converting this fake black input from byte to float,
and then converting it to sequencer color space.
Test case of two EXR files blended over each other, at 4K resolution,
on Ryzen 5950X (Win10/VS2022):
- whole sequencer_preview_region_draw 115ms -> 75ms
- seq_render_give_ibuf part: 64ms -> 24ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127310
Speedup the Color Balance VSE strip modifier, with two things:
- Generally, use a much lower overhead parallel_for, also with
lower grain size (32 image rows, instead of 64 that were used
before). This is what makes the "float" variant faster.
- For "byte" variant, create a precalculated lookup table instead
of doing all the math per-pixel. This was *almost* done in
existing code, except it was put into the code path that was
never-ever used. However, since this is all done on premultiplied
values, I'm using lookup table size of 1024 instead of 256, so
that semitransparent pixels get some more precision for
"in-between values". This LUT is what results in the main speedup
of "byte" variant.
Calculating Color Balance at 4K resolution, times in milliseconds:
- PC (Ryzen 5950X), PNG (byte): 22.2 -> 2.9 ms, EXR (float): 20.1 -> 15.2 ms
- Mac (M1 Max), PNG (byte): 28.9 -> 7.5 ms, EXR (float): 21.8 -> 8.5 ms
More timing details in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127121
VSE timeline, when many (hundreds/thousands) of thumbnails were visible, was
very slow to redraw. This PR makes them 3-10x faster to redraw, by stopping
doing things that are slow :) Part of #126087 thumbnail improvements task.
- No longer do mute semitransparency or corner rounding on the CPU, do it in
shader instead.
- Stop creating a separate GPU texture for each thumbnail, on every repaint,
and drawing each thumbnail as a separate draw call. Instead, put thumbnails
into a single texture atlas (using a simple shelf packing algorithm), and
draw them in batch, passing data via UBO. The atlas is still re-created every
frame, but that does not seem to be a performance issue. Thumbnails are
cropped horizontally based on how much of their parts are visible (e.g. a
narrow strip on screen), so realistically the atlas size is kinda
proportional to screen size, and ends up being just several megabytes of data
transfer between CPU -> GPU each frame.
On this Sprite Fright edit timeline view (612 visible thumbnails), time taken
to repaint the timeline window:
- Mac (M1 Max, Metal): 68.1ms -> 4.7ms
- Windows (Ryzen 5950X, RTX 3080Ti, OpenGL): 23.7ms -> 6.8ms
This also fixes a visual issue with thumbnails, where when strips are very
tall, the "rounded corners" that were poked right into the thumbnail bitmap
on the CPU were showing up due to actual bitmap being scaled up a lot.
Pull Request: https://projects.blender.org/blender/blender/pulls/126972
Implementing part of design outlined in #126087.
- VSE thumbnail cache has a new implementation, hopefully simpler
and easier to understand.
- Instead of cache key being a VSE strip, frame index, plus
complicated logic for cache items linking etc.,
- The cache is keyed by media file path (if multiple strips
use the same input file, they will share cache entries), frame
index within media file, and any extra data (e.g. steam index
for multi-steam videos)
- Much reduced cache flickering and strange/weird thumbnail choices.
- Likewise, thumbnails no longer disappear-and-reload on operations
like Undo, dragging new video strip into timeline, or F12 render.
- Thumbnails now load faster.
- Images use dedicated/faster thumbnail loading routines when a
format can do that (e.g. JPG and EXR can).
- Movies reuse ffmpeg decoding context for neighboring strips
that use the same file (as often happens when cutting footage)
- Thumbnail requests are processed on several threads now too.
Images and more detail in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126405
Adds the ability to connect and disconnect strips in the VSE.
- Connected strips have an icon indicating their status, and attempting
to select one connected strip selects all other connected strips in
that chain.
- If the user attempts to connect a strip that is already connected to
other strips, that strip will disconnect itself from others before
connecting to new strips.
- Preview selection also works in bulk if multiple video strips are
connected together in the timeline.
- When adding new strips from the Add menu or the File Browser, strips
from the same file are connected by default. There's an option to
turn this off in Editing > Video Sequencer user preferences.
- It is possible to individually tweak strips/handles and ignore
connections with Alt+Click.
- This shortcut overrides the old keymap item for "Linked Handle"
selection. The property still exists if people want to use that
shortcut for its old purpose.
- To make sure that connections remain valid even after duplication,
I've added a condition to `seq_new_fix_links_recursive` that also
updates connections using the `seq->tmp` var. (A note -- I've updated
the comment for this field in `DNA_sequence_types.h` because the var
is only used for duplication now. It was once present in
`select_more_less_seq__internal` to be used for linked selection but
is gone now).
- There are also functions to cut one-way links and make sure that
all strips are bidirectionally connected after duplicating.
Pull Request: https://projects.blender.org/blender/blender/pulls/124333
API: merged IMB_scalefastImBuf, IMB_scaleImBuf, IMB_scaleImBuf_threaded
into one function IMB_scale with enum IMBScaleFilter {Nearest, Bilinear, Box}
and bool "threaded" param.
Performance:
- Box filtering (nee IMB_scaleImBuf) can be multi-threaded now.
- Nearest filtering (nee IMB_scalefastImBuf) can be multi-threaded now.
Also fix performance regression on float images caused by fix in #126234
- Bilinear filtering (nee IMB_scaleImBuf_threaded) is several times faster now.
Correctness:
- Nearest and Box filtering: no longer loses half of edge pixels when scaling
up.
- Box: fixed garbage results (and possible out of bounds reads) for non-4
channel float images.
- Bilinear: no longer shifts image when scaling up.
- Bilinear: properly filters when scaling down by 2x2.
Test coverage:
- Add gtest coverage for various IMB_scale modes.
- Add a IMB_performance_test performance test, ran manually.
More details, images and performance numbers in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126390
SEQ_transform_sequence_can_be_translated, SEQ_transform_single_image_check,
SEQ_can_use_proxy can all trivially take const Sequence argument.
Made SEQ_give_frame_index take const Sequence too, by removing the
"modify seq strobe to be at least 1.0" code. The strobe itself is only
ever used inside the same function, and is already guarded by
"is strobe > 1.0" check.
Original code seems to be coming all the way from 2009 with
commit message "2.5. 12k lines of sequencer back!" (03fc5696dc).
Pull Request: https://projects.blender.org/blender/blender/pulls/126021
Prevent VSE cache returning image when running render job.
Previously, before render job was started, cache was cleared, but since
disk cache is not cleared, it returned an image. Now cache is not
cleared, but it does not return or store images when rendering.
Pull Request: https://projects.blender.org/blender/blender/pulls/125947
Match function and declaration names, picking names based on
consistency with related code & clarity.
Also changes for old conventions, missed in previous cleanups:
- name -> filepath
- tname -> newname
- maxlen -> maxncpy
As well as takes text color transparency into account. That is, text
"shadow" now virtually casts shadow of however the text itself + outline
looks like. The result is blurred, and extra shadow color tint and
transparency is applied.
Performance testing: seems to be slightly faster than before in my tests.
Probably because the shadow "image" that is now blurred only contains
one byte per pixel (the transparency), instead of full uchar4 per pixel
(with only the "transparency" being blurred).
I have extended sequencer render tests to better cover various text
outline/shadow/box combinations.
Pull Request: https://projects.blender.org/blender/blender/pulls/125445
This is only visible when text color has transparency or is fully
transparent. Desired behavior is that the outline would only be
"outside" the actual text.
Pull Request: https://projects.blender.org/blender/blender/pulls/125372
The cache invalidation worked on basis of timeline frame instead of
frame index, which caused, that these images were outside of range, that
was considered to be changed, after strip was moved.
In this case the issue happened only with effect strips. This was not
an issue before, mainly because raw images for these strips were not
stored. For other strip types, raw image does not usually change.
This seems to be "caused" by f4f708a54f .
Pull Request: https://projects.blender.org/blender/blender/pulls/124870
If a strip had effect strips applied on it, and using the reset
retiming operator caused the strips to increase in length, the effect
strips would become overlapped with any strips they hit, unless the
"Expand" overlap mode was used.
This is because the "effect chain" strips weren't included
in #strips.
Co-authored-by: Richard Antalik <richardantalik@gmail.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/124750
