If conditions are not met for offscreen rendering, render job is used as
fallback. This can be skipped if there is possibility of race condition.
This was only considered for case where `context->for_render` was false,
since this indicates that call is invoked by the job itself.
If scene strip display is set to `OB_RENDER`, it needs to start the
render job. So variable `do_seq_gl` was renamed to `is_preview` and
reused in race avoidance connditon. This was done to convey clearer
meaning and match comments.
This change does make code less safe - race is avoided by UI not
allowing to run F12 render and opengl render at the same time. However,
I don't think, this could be easily resolved on VSE side without
suspending thread or being able to run multiple render jobs.
Pull Request: https://projects.blender.org/blender/blender/pulls/127926
When I was learning VSE code, MAXSEQ constant (a preprocessor define!) was
confusing. It makes it sound like it is "max sequences", but it is actually
"max channels". So rename it to SEQ_MAX_CHANNELS and make it C++ constexpr int
instead of preprocessor macro.
Pull Request: https://projects.blender.org/blender/blender/pulls/128024
To avoid unnecessary looping over listbase items the function
`BLI_listbase_count_at_most` was used however it resulting in an awkward
expression: `BLI_listbase_count_at_most(list, count + 1) == count`
replace this with `BLI_listbase_count_is_equal_to(list, count)`.
When doing average/min/max luminance calculations for tonemapping, ignore
pixels that are outside of the strip rectangle. Due to how VSE innards work,
when a strip is positioned to not cover the whole screen, "the rest" is filled
with transparent black. For tonemapping, this was dragging average calculated
luminance way down.
Images of the issue in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127207
Image/movie strip controls color controls: "Saturation" and "Multiply" were
both single-threaded code. Multi-thread both of them. Timings to apply them
on a 4K resolution byte image, on Ryzen 5950X (Win10/VS2022):
- Saturation: 97ms -> 6.3ms
- Multiply: 11.5ms -> 1.1ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127409
SEQ_modifier_apply_stack is called from exactly one place, and it always
was throwing away the input image, and taking the newly produced image.
Which means, we can stop copying it. Just run the modifiers on the
input image.
In a test file that is HD (1080p) resolution, two EXR image sequences,
adjustment layer with Color Balance, and a text strip on top,
playback framerate (Ryzen 5950X, Win10/VS2022) goes 20.8 -> 22.1 fps
Pull Request: https://projects.blender.org/blender/blender/pulls/127346
When a strip with alpha-over blend mode (which is default) has an alpha
channel and is at the bottom of the whole stack, there's no point in
blending it with black color; the result will be identical.
So stop doing that. Also, whenever some other case happens at the bottom
of the stack (e.g. some other blend mode), create the fake "black input"
image at the needed color type (float or byte) and with the correct
color space already applied. Otherwise, especially for floats, VSE
spends much time converting this fake black input from byte to float,
and then converting it to sequencer color space.
Test case of two EXR files blended over each other, at 4K resolution,
on Ryzen 5950X (Win10/VS2022):
- whole sequencer_preview_region_draw 115ms -> 75ms
- seq_render_give_ibuf part: 64ms -> 24ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127310
Implementing part of design outlined in #126087.
- VSE thumbnail cache has a new implementation, hopefully simpler
and easier to understand.
- Instead of cache key being a VSE strip, frame index, plus
complicated logic for cache items linking etc.,
- The cache is keyed by media file path (if multiple strips
use the same input file, they will share cache entries), frame
index within media file, and any extra data (e.g. steam index
for multi-steam videos)
- Much reduced cache flickering and strange/weird thumbnail choices.
- Likewise, thumbnails no longer disappear-and-reload on operations
like Undo, dragging new video strip into timeline, or F12 render.
- Thumbnails now load faster.
- Images use dedicated/faster thumbnail loading routines when a
format can do that (e.g. JPG and EXR can).
- Movies reuse ffmpeg decoding context for neighboring strips
that use the same file (as often happens when cutting footage)
- Thumbnail requests are processed on several threads now too.
Images and more detail in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126405
This commit moves generated `RNA_blender.h`, `RNA_prototype.h` and
`RNA_blender_cpp.h` headers to become C++ header files.
It also removes the now useless `RNA_EXTERN_C` defines, and just
directly use the `extern` keyword. We do not need anymore `extern "C"`
declarations here.
Pull Request: https://projects.blender.org/blender/blender/pulls/124469
Code that implemented "show missing media in magenta" in VSE preview
only implemented that for images and movies, and forgot that scene
strips can also be missing.
Pull Request: https://projects.blender.org/blender/blender/pulls/124124
Caused by incorrect output of `SEQ_give_frame_index` for effect strips.
Also since output value is float, it has to be truncated to integer,
when image bufferes are rendered for the effect.
Pull Request: https://projects.blender.org/blender/blender/pulls/123914
When preview is downscaled and transformation origin is not the center
of the image, this causes unexpected offset. This happened, because one
matrix combined image downscaling, so it fits into preview and user
defined scale. When origin was not center of the image, this results in
incorrect offset.
Solved by splitting 1 matrix in `sequencer_image_crop_transform_matrix`
into 2 matrices. First matrix just centers and scales image to expected
size. Second matrix performs rest of transform operations. This code is
bit easier to read as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/123776
The issue was caused by the ImBuf of the scene strip render sharing the float
buffer pixels with the ImBuf from the render result. If the render result is
ever gets freed (i.e., by a request to perform another render) it'll leave the
strip ImBuf pointing to a freed memory.
This was caused by the #109788.
The simple solution is to restore the code to the state prior to the ImBuf
refactor in the RenderRsult. A better solution would be to use implicit
sharing, similar to how it was done in the #108045.
Pull Request: https://projects.blender.org/blender/blender/pulls/123731
A strip was wrongly deemed to be opaque (and thus strips below it
were skipped from rendering), if the strip content was opaque, but
it gained transparency via presence of a Mask modifier.
While at it, I also noticed that a strip could have been wrongly deemed
opaque when it had a Multiplier < 1.0 with "multiply alpha" option
checked under strip Color settings. So fixed that too.
The whole logic of that factored out into is_opaque_alpha_over function
to make the call site clearer.
Pull Request: https://projects.blender.org/blender/blender/pulls/123303
Sequencer timeline displays red tint & appropriate icons for strips that are
missing media file (images, movies, audio, or meta strips that contain such).
Sequencer preview and rendering displays missing media strips as magenta,
similar to missing textures elsewhere in Blender. This is on by default,
sequencer view settings have an option to turn it off.
Pull Request: https://projects.blender.org/blender/blender/pulls/116869
Ultimately this was due to missing nullptr check before trying to
process the image, but this should have been caught when loading
ImBufAnims. If any is missing, cancel multiview loading and load
movie as if it was single view only.
Often in previs setups, you have several "variations" of image/movie strips
for review, where the topmost one completely covers the others under it. VSE
rendering already had an optimization where if there's a fully opaque strip
that covers the whole screen, then the lower strips are skipped from
processing/rendering. However, it was not handling the case of non-fullscreen
strips (e.g. you'd have a Color strip near top & bottom for letterboxing, and
an opaque strip "in the middle").
This adds a simple "occluder tracking", and skips strips that are completely
covered by any single opaque strip that is above it (like outlined in #117790
task).
Playback of Gold previs between 1:42 and 1:55, on Windows/Ryzen5950X:
- Average frame time 28.5ms -> 23.8ms
- Median frame time 24.1ms -> 21.5ms
- Two slowest frames: 263->189ms, 194->178ms
Rendering the Gold previs movie: 325s -> 304s (93% of previous time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118396
In some/many cases, an `ImBuf` is allocated, and all the pixels are
immediately filled by some code. Doing the memory clear within allocation
is just memory traffic for no good reason.
Add a flag to skip initialization of ImBuf pixels (IB_uninitialized_pixels)
and use that in some parts of VSE effects/rendering/cache/scopes, as well
as image loading code.
Rendering out VSE movie, on Windows/VS2022/Ryzen5950X:
- Sprite Fright: 443sec -> 414sec (takes 93% of previous time)
- Gold previs: 367sec -> 325sec (takes 88% of prev time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118321
Part of "improve filtering situation" (#116980), now strip scaling filter defaults to "Auto" which has logic like:
- No scale, no rotation, integer positions: Nearest (fastest)
- Scaling up by more than 2x: Cubic Mitchell, so you get nicer blending between pixels than with bilinear,
- Scaling down by more than 2x: Box, so that many pixels are averaged properly without too much aliasing,
- Otherwise: Bilinear
Existing strips that use Bilinear (which is default) get switched to Auto when loading older files.
All of this has an advantage that unless you have some special needs for your look, you can leave it at default and it will look decently good at either large up-scaling or large down-scaling, but not waste performance if you don't use any scaling at all. Previously none of the choices were good in "all cases": box (née subsampled3x3) only looks good when scaling down, cubic only looks good when scaling up, default bilinear leaves performance on the table when you don't use any scale/rotation, etc.
On something like Gold movie current edit, most of the strips effectively use Nearest now, except some that are translated into non-integer pixel positions; those stay effectively Bilinear.
Pull Request: https://projects.blender.org/blender/blender/pulls/117853
- enum class StripEarlyOut instead of raw integer defines
- SeqRenderState default initializer instead of seq_render_state_init
- Vector<Sequence*> instead of manually sized arrays of pointers
- some const to several function arguments
Pull Request: https://projects.blender.org/blender/blender/pulls/117829
Part of overall "improve filtering situation" (#116980): replace Subsampled3x3
(added for blender 3.5 in f210842a72 et al.) strip scaling filter with a
general Box filter.
Subsampled3x3 is really a Box filter ("average pixel values over NxM region"),
hardcoded to 3x3 size. As such, it works pretty well when downscaling images by
3x on each axis. But starts to break down and introduce aliasing at other
scaling factors. Also when scaling up or scaling down by less than 3x, using
total of 9 samples is a bit of overkill and hurts performance.
So instead, calculate the amount of NxM samples needed by looking at scaling
factors on X/Y axes. Note: use at least 2 samples on each axis, so that when
rotation is present, the result edges will get some anti-aliasing, just like it
was happening in previous filter implementation.
Images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117584
Part of overall "improve filtering situation" (#116980) task:
Add "Cubic Mitchell" filtering option to VSE strips. This is a cubic (4x4)
filter that generally looks better than bilinear, while not blurring the image
as much as the Cubic BSpline filter that exists elsewhere within Blender. It is
also default in many other apps.
Rename the (very recently added) VSE Bicubic filter option to Cubic BSpline.
Images in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/117517
If a scene sequencer strip references a scene with no composite output,
it will crash due to null image buffer output. This patch fixes that by
allocating an empty buffer in those cases.
Pull Request: https://projects.blender.org/blender/blender/pulls/117135
Part of overall "improve filtering situation" (#116980) task:
* Add Bicubic filtering option to strip Transform "Filter" setting.
Previously this option only existed in Transform Effect "Interpolation"
setting.
- With this addition, it feels like the transform effect could
possibly be marked as legacy/deprecated, since the regular Transform
that is on all strips can do everything that Transform Effect did?
* Speed up bicubic filtering (used now in VSE, but also in CPU Compositor,
image paint, etc.) by slightly simplifying the code and using some SIMD.
Upscaling 96x54 image to 3840x2160 resolution, using Bicubic filtering:
- Windows (VS2022, Ryzen 5950X): 35.5ms -> 15.1ms
- Mac (clang 15, M1 Max): 29.6ms -> 24.4ms
* Add gtest coverage for bicubic functionality.
Pull Request: https://projects.blender.org/blender/blender/pulls/117100
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
Since b1526dd2c6, viewport renderer sets `G.is_rendering`, but VSE scene
rendering function used this to decide whether to do offscreen opengl
render or use render API. This logic was quite weak.
Use `SeqRenderData::for_render` instead of `G.is_rendering`, since it
explicitly defines whether strip rendering is invoked by F12 render job.
Since offscreen gl rendering rely on `BLI_thread_is_main()` being
true, use this to initialize `do_seq_gl` variable for clarity.
The use of render job path was further conditioned on `G.is_rendering`,
with exception of running headless, but this condition was incorrect.
This condition was reformulated as precondition, which if true, returns
`nullptr` instead of crashing.
Pull Request: https://projects.blender.org/blender/blender/pulls/115079
Use `VectorSet`, `Vector` or `Span` instead of `SeqCollection` struct.
It is now possible to use native `for` loops and `SEQ_ITERATOR_FOREACH`
macro can be removed.
Another feature is, sets of strips no longer needs to be freed. However,
this poses a limitation, that query functions can not be used in case,
where these sets need to be available outside of scope where they are
created.
Pull Request: https://projects.blender.org/blender/blender/pulls/111909
After eda58d6419, multiply operation does not affect alpha channel, but
Some users do expect this feature present.
This adds option `multiply_alpha`, so that multiplication (in strip
color panel) will affect alpha channel as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/113791
Conversion from timeline frame to frame index was done by casting to
integer, which followed logic of ffmpeg seeking. However this is not
best approach in some cases - for example when FPS of scene and movie
differs by a very small amount. In this case the first frame could be
duplicated and all other frames will appear as offset by one frame.
In particular this may happen scene is set to 29.97 fps and movie is
encoded at 30000/1001 fps. A frame will still have to be duplicated, but
it should be frame where decimal of frame index crosses 0.5 to keep
audio and video in sync as best as possible.
Pull Request: https://projects.blender.org/blender/blender/pulls/113870
