Move `Library.runtime` to be a pointer, move the related
`LibraryRuntime` struct to `BKE_library.hh`. Similar to e.g.
Mesh.runtime, that pointer is expected to always be valid, and is
allocated at readtime or when creating a new Library ID.
Related smaller changes:
* Write code now uses standard ID writing codepath for Library IDs too.
* Runtime pointer is reset to nullptr before writing.
* Looking up a library by its absolute path is now handled through a
dedicated utils, `search_filepath_abs`, instead of using
`BLI_findstring`.
Pull Request: https://projects.blender.org/blender/blender/pulls/134188
10/12bpp movie writing code assumes that input image is properly
floating point for these cases. That is always true for regular
rendering, however VSE can produce 8bpp images, if there
are no float/HDR strips in there. Make these convert into a float
image in the render pipeline when needed.
Pull Request: https://projects.blender.org/blender/blender/pulls/133802
This PR explicitly declares integer type for `image_center_offs`, which
formalizes accidentally implemented correct behavior. Previously type
was float, all rhs types in expression were integers, so result was int
cast to float.
Comment is also added clarifying why integer must be used - float may
cause images to be interpolated, even when translation is set to 0.
There should be no functional changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/133573
The rest of blender does handle multi-layer EXR images, using the
"combined" or RGBA/RGB layers when the visual result is needed. Make
VSE do the same.
While fixing this, I found several issues in other not well tested code
and had to fix them:
- IMB_buffer_float_from_float_threaded was wrongly using source channels
as destination channels, producing garbage result.
- IMB_scale_into_new was not assigning channels to destination image.
Pull Request: https://projects.blender.org/blender/blender/pulls/132790
Depending on proxy size and scaling factor, the "does proxy cover
whole screen" code was producing wrong results, due to mismatched
resolutions it was working on: SeqRenderData at that point contains
proxy resolution (reduced), but SEQ_image_transform_final_quad_get
works at full resolution. The produced quad needs to be scaled down
to what the render context is operating at.
This renames the struct `Sequence` to `Strip`.
While the motivation for this partially comes from
the "Sequence Design" #131329, it seems like this
is a good refactor whether the design gets implemented
or not.
The `Sequence` represents what users see as strips in the
VSE. Many places in the code already refere to a `Sequence`
as "strip". It's the C-style "base class" of all strip types.
This also renames the python RNA type `bpy.types.Sequence`
to `bpy.types.Strip` which means that this technically breaks
the python API.
Pull Request: https://projects.blender.org/blender/blender/pulls/132179
Cleanup (and make slightly faster as a side effect) the way VSE effects
do multi-threading. Previously (some of them) were using
IMB_processor_apply_threaded with C-like machinery (which internally
uses a task pool), switch that over to a helper apply_effect_op
(which internally uses a parallel for). Based on profiling, parallel
for is slightly more efficient (task pool takes a bit until all the
tasks are "pushed" into the pool). Note however that some VSE effects
were already doing parallel for internally; these are not affected.
VSE scene at 4K resolution, with four 4K resolution PNG images blended
over each other, time it takes to do render_strip_stack:
- Ryzen 5950X (Win/VS2022): 38.9ms -> 34.7ms
- Mac M4 Max: 21.9ms -> 19.8ms
Now that all VSE effects are internally threaded via parallel for,
there's no need for the init_execution and execute_slice machinery,
so remove all that.
You might also notice that half of "over drop" effect code is gone.
It was accidentally not doing anything whatsoever for the last 18 years
(since 2.42), and currently observed behavior matches documentation
and "internet knowledge", so let's accept it as correct.
Pull Request: https://projects.blender.org/blender/blender/pulls/132380
Part of #130975: split off overly large VSE effects.cc (almost 4000
lines) into separate source files. No behavior changes.
- Add / Subtract / Multiply all go into one source file,
- Blend Over / Blend Under / Over Drop / other blend modes all go
into one source file,
- Cross and Gamma Cross go into one source file,
- All others go into their own files.
Pull Request: https://projects.blender.org/blender/blender/pulls/132323
This renames the `Strip` struct to `StripData` and also renames
the `Sequence::strip` member to `Sequence::data`.
This is a first step towards naming the `Sequence` struct to `Strip`.
Pull Request: https://projects.blender.org/blender/blender/pulls/132165
- All movie related public headers now have MOV_ prefix instead of
IMB_movie_.
- All movie related public functions now have MOV_ prefix as well,
instead of IMB_movie_ or IMB_anim_.
- IMB_anim.hh -> MOV_read.hh (also ImBufAnim -> MovieReader), and
various utility functions not related to playback were split off
into MOV_util.hh.
- Other function name tweaks for clarity, e.g. IMB_suffix_anim
-> MOV_set_multiview_suffix and so on.
- All except one usages of MOV_get_fps (nee IMB_anim_get_fps) were
ultimately just converting returned value into a float. So make
MOV_get_fps just return that directly. For the (exactly just one)
place that needs numerator and denominator, have
MOV_get_fps_num_denom.
- Code comments on the public header functions.
- Removed never-used code paths inside movie timecode proxy building
file.
It might be easier to review each commit separately.
Pull Request: https://projects.blender.org/blender/blender/pulls/132145
Previously, code related to reading/writing movie files via ffmpeg was
scattered around: some under blenkernel, some directly in generic
imbuf headers, some under intern/ffmpeg. Some of the files were named
with not exactly clear names. Some parts not directly related to movies
were including ffmpeg headers directly (rna_scene.cc).
What is in this PR:
Movie and ffmpeg related code is now under imbuf/movie:
- IMB_anim.hh: movie reading, proxy querying, various utility functions.
- IMB_movie_enums.hh: simple enum definitions,
- IMB_movie_write.hh: movie writing functions.
- intern: actual implementation and private headers.
- ffmpeg_compat.h: various ffmpeg version difference handling
utilities,
- ffmpeg_swscale.hh/cc: scaling and format conversion utilities
for ffmpeg libswscale,
- ffmpeg_util.hh/cc: misc utilities related to ffmpeg,
- movie_proxy_indexer.hh/cc: proxies and timecode indexing for movies,
- movie_read.hh/cc: decoding of movies into images,
- movie_write.cc: encoding of images into movies.
- tests: basic ffmpeg library unit tests that previously
lived under intern/ffmpeg.
Interface changes (at C++ level, no Python API changes):
- Mostly just movie related functions that were BKE_ previously, are now IMB_.
- I did one large-ish change though, and that is to remove bMovieHandle
struct that had pointers to several functions. Now that is
IMB_movie_write_begin, IMB_movie_write_append, IMB_movie_write_end
functions using a single opaque struct handle. As a result, usages
of that in pipeline.cc and render_opengl.cc have changed.
Pull Request: https://projects.blender.org/blender/blender/pulls/132074
NOTE: This also required some changes to Cycles code itself, who is now
directly including `BKE_image.hh` instead of declaring a few prototypes
of these functions in its `blender/utils.h` header (due to C++ functions
names mangling, this was not working anymore).
Pull Request: https://projects.blender.org/blender/blender/pulls/130174
This extends the `Vector` API to support transfering ownership of a memory
buffer to and from a `Vector`. This reduces the need for unnecessary copies in
some cases which converting between data-structures. A new
`VectorSet::extract_vector` method is added that takes O(1) time. Previously,
this was only possible in O(n) time by copying the entire array.
The `Vector::release` method can be used to e.g. build a vector with the C++
container, but then extract it for use in DNA data.
Pull Request: https://projects.blender.org/blender/blender/pulls/129736
If conditions are not met for offscreen rendering, render job is used as
fallback. This can be skipped if there is possibility of race condition.
This was only considered for case where `context->for_render` was false,
since this indicates that call is invoked by the job itself.
If scene strip display is set to `OB_RENDER`, it needs to start the
render job. So variable `do_seq_gl` was renamed to `is_preview` and
reused in race avoidance connditon. This was done to convey clearer
meaning and match comments.
This change does make code less safe - race is avoided by UI not
allowing to run F12 render and opengl render at the same time. However,
I don't think, this could be easily resolved on VSE side without
suspending thread or being able to run multiple render jobs.
Pull Request: https://projects.blender.org/blender/blender/pulls/127926
When I was learning VSE code, MAXSEQ constant (a preprocessor define!) was
confusing. It makes it sound like it is "max sequences", but it is actually
"max channels". So rename it to SEQ_MAX_CHANNELS and make it C++ constexpr int
instead of preprocessor macro.
Pull Request: https://projects.blender.org/blender/blender/pulls/128024
To avoid unnecessary looping over listbase items the function
`BLI_listbase_count_at_most` was used however it resulting in an awkward
expression: `BLI_listbase_count_at_most(list, count + 1) == count`
replace this with `BLI_listbase_count_is_equal_to(list, count)`.
When doing average/min/max luminance calculations for tonemapping, ignore
pixels that are outside of the strip rectangle. Due to how VSE innards work,
when a strip is positioned to not cover the whole screen, "the rest" is filled
with transparent black. For tonemapping, this was dragging average calculated
luminance way down.
Images of the issue in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/127207
Image/movie strip controls color controls: "Saturation" and "Multiply" were
both single-threaded code. Multi-thread both of them. Timings to apply them
on a 4K resolution byte image, on Ryzen 5950X (Win10/VS2022):
- Saturation: 97ms -> 6.3ms
- Multiply: 11.5ms -> 1.1ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127409
SEQ_modifier_apply_stack is called from exactly one place, and it always
was throwing away the input image, and taking the newly produced image.
Which means, we can stop copying it. Just run the modifiers on the
input image.
In a test file that is HD (1080p) resolution, two EXR image sequences,
adjustment layer with Color Balance, and a text strip on top,
playback framerate (Ryzen 5950X, Win10/VS2022) goes 20.8 -> 22.1 fps
Pull Request: https://projects.blender.org/blender/blender/pulls/127346
When a strip with alpha-over blend mode (which is default) has an alpha
channel and is at the bottom of the whole stack, there's no point in
blending it with black color; the result will be identical.
So stop doing that. Also, whenever some other case happens at the bottom
of the stack (e.g. some other blend mode), create the fake "black input"
image at the needed color type (float or byte) and with the correct
color space already applied. Otherwise, especially for floats, VSE
spends much time converting this fake black input from byte to float,
and then converting it to sequencer color space.
Test case of two EXR files blended over each other, at 4K resolution,
on Ryzen 5950X (Win10/VS2022):
- whole sequencer_preview_region_draw 115ms -> 75ms
- seq_render_give_ibuf part: 64ms -> 24ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127310
Implementing part of design outlined in #126087.
- VSE thumbnail cache has a new implementation, hopefully simpler
and easier to understand.
- Instead of cache key being a VSE strip, frame index, plus
complicated logic for cache items linking etc.,
- The cache is keyed by media file path (if multiple strips
use the same input file, they will share cache entries), frame
index within media file, and any extra data (e.g. steam index
for multi-steam videos)
- Much reduced cache flickering and strange/weird thumbnail choices.
- Likewise, thumbnails no longer disappear-and-reload on operations
like Undo, dragging new video strip into timeline, or F12 render.
- Thumbnails now load faster.
- Images use dedicated/faster thumbnail loading routines when a
format can do that (e.g. JPG and EXR can).
- Movies reuse ffmpeg decoding context for neighboring strips
that use the same file (as often happens when cutting footage)
- Thumbnail requests are processed on several threads now too.
Images and more detail in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/126405
This commit moves generated `RNA_blender.h`, `RNA_prototype.h` and
`RNA_blender_cpp.h` headers to become C++ header files.
It also removes the now useless `RNA_EXTERN_C` defines, and just
directly use the `extern` keyword. We do not need anymore `extern "C"`
declarations here.
Pull Request: https://projects.blender.org/blender/blender/pulls/124469
Code that implemented "show missing media in magenta" in VSE preview
only implemented that for images and movies, and forgot that scene
strips can also be missing.
Pull Request: https://projects.blender.org/blender/blender/pulls/124124
Caused by incorrect output of `SEQ_give_frame_index` for effect strips.
Also since output value is float, it has to be truncated to integer,
when image bufferes are rendered for the effect.
Pull Request: https://projects.blender.org/blender/blender/pulls/123914
When preview is downscaled and transformation origin is not the center
of the image, this causes unexpected offset. This happened, because one
matrix combined image downscaling, so it fits into preview and user
defined scale. When origin was not center of the image, this results in
incorrect offset.
Solved by splitting 1 matrix in `sequencer_image_crop_transform_matrix`
into 2 matrices. First matrix just centers and scales image to expected
size. Second matrix performs rest of transform operations. This code is
bit easier to read as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/123776
The issue was caused by the ImBuf of the scene strip render sharing the float
buffer pixels with the ImBuf from the render result. If the render result is
ever gets freed (i.e., by a request to perform another render) it'll leave the
strip ImBuf pointing to a freed memory.
This was caused by the #109788.
The simple solution is to restore the code to the state prior to the ImBuf
refactor in the RenderRsult. A better solution would be to use implicit
sharing, similar to how it was done in the #108045.
Pull Request: https://projects.blender.org/blender/blender/pulls/123731
A strip was wrongly deemed to be opaque (and thus strips below it
were skipped from rendering), if the strip content was opaque, but
it gained transparency via presence of a Mask modifier.
While at it, I also noticed that a strip could have been wrongly deemed
opaque when it had a Multiplier < 1.0 with "multiply alpha" option
checked under strip Color settings. So fixed that too.
The whole logic of that factored out into is_opaque_alpha_over function
to make the call site clearer.
Pull Request: https://projects.blender.org/blender/blender/pulls/123303
Sequencer timeline displays red tint & appropriate icons for strips that are
missing media file (images, movies, audio, or meta strips that contain such).
Sequencer preview and rendering displays missing media strips as magenta,
similar to missing textures elsewhere in Blender. This is on by default,
sequencer view settings have an option to turn it off.
Pull Request: https://projects.blender.org/blender/blender/pulls/116869
Ultimately this was due to missing nullptr check before trying to
process the image, but this should have been caught when loading
ImBufAnims. If any is missing, cancel multiview loading and load
movie as if it was single view only.
Often in previs setups, you have several "variations" of image/movie strips
for review, where the topmost one completely covers the others under it. VSE
rendering already had an optimization where if there's a fully opaque strip
that covers the whole screen, then the lower strips are skipped from
processing/rendering. However, it was not handling the case of non-fullscreen
strips (e.g. you'd have a Color strip near top & bottom for letterboxing, and
an opaque strip "in the middle").
This adds a simple "occluder tracking", and skips strips that are completely
covered by any single opaque strip that is above it (like outlined in #117790
task).
Playback of Gold previs between 1:42 and 1:55, on Windows/Ryzen5950X:
- Average frame time 28.5ms -> 23.8ms
- Median frame time 24.1ms -> 21.5ms
- Two slowest frames: 263->189ms, 194->178ms
Rendering the Gold previs movie: 325s -> 304s (93% of previous time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118396
In some/many cases, an `ImBuf` is allocated, and all the pixels are
immediately filled by some code. Doing the memory clear within allocation
is just memory traffic for no good reason.
Add a flag to skip initialization of ImBuf pixels (IB_uninitialized_pixels)
and use that in some parts of VSE effects/rendering/cache/scopes, as well
as image loading code.
Rendering out VSE movie, on Windows/VS2022/Ryzen5950X:
- Sprite Fright: 443sec -> 414sec (takes 93% of previous time)
- Gold previs: 367sec -> 325sec (takes 88% of prev time)
Pull Request: https://projects.blender.org/blender/blender/pulls/118321
