Since one user-defined conversion operator is allowed during implicit conversion,
and after this conversion here is a constructor which can accept result
of conversion, there was a backdoor for a vector types to up-cast their
dimensions via cast to pointer type of a component of a vector. Since it was
implicit and non-intentional it led to buffer overflows.
Pull Request: https://projects.blender.org/blender/blender/pulls/132927
* Ensure valid bit depth is set along with file type
* Guard against invalid inputs in stereo imbuf creation
* Remove some unused code
Thanks Yiming Wu for finding the cause.
Pull Request: https://projects.blender.org/blender/blender/pulls/133499
The rest of blender does handle multi-layer EXR images, using the
"combined" or RGBA/RGB layers when the visual result is needed. Make
VSE do the same.
While fixing this, I found several issues in other not well tested code
and had to fix them:
- IMB_buffer_float_from_float_threaded was wrongly using source channels
as destination channels, producing garbage result.
- IMB_scale_into_new was not assigning channels to destination image.
Pull Request: https://projects.blender.org/blender/blender/pulls/132790
When using clangd or running clang-tidy on headers there are
currently many errors. These are noisy in IDEs, make auto fixes
impossible, and break features like code completion, refactoring
and navigation.
This makes source/blender headers work by themselves, which is
generally the goal anyway. But #includes and forward declarations
were often incomplete.
* Add #includes and forward declarations
* Add IWYU pragma: export in a few places
* Remove some unused #includes (but there are many more)
* Tweak ShaderCreateInfo macros to work better with clangd
Some types of headers still have errors, these could be fixed or
worked around with more investigation. Mostly preprocessor
template headers like NOD_static_types.h.
Note that that disabling WITH_UNITY_BUILD is required for clangd to
work properly, otherwise compile_commands.json does not contain
the information for the relevant source files.
For more details see the developer docs:
https://developer.blender.org/docs/handbook/tooling/clangd/
Pull Request: https://projects.blender.org/blender/blender/pulls/132608
This renames the struct `Sequence` to `Strip`.
While the motivation for this partially comes from
the "Sequence Design" #131329, it seems like this
is a good refactor whether the design gets implemented
or not.
The `Sequence` represents what users see as strips in the
VSE. Many places in the code already refere to a `Sequence`
as "strip". It's the C-style "base class" of all strip types.
This also renames the python RNA type `bpy.types.Sequence`
to `bpy.types.Strip` which means that this technically breaks
the python API.
Pull Request: https://projects.blender.org/blender/blender/pulls/132179
This renames the `Strip` struct to `StripData` and also renames
the `Sequence::strip` member to `Sequence::data`.
This is a first step towards naming the `Sequence` struct to `Strip`.
Pull Request: https://projects.blender.org/blender/blender/pulls/132165
- All movie related public headers now have MOV_ prefix instead of
IMB_movie_.
- All movie related public functions now have MOV_ prefix as well,
instead of IMB_movie_ or IMB_anim_.
- IMB_anim.hh -> MOV_read.hh (also ImBufAnim -> MovieReader), and
various utility functions not related to playback were split off
into MOV_util.hh.
- Other function name tweaks for clarity, e.g. IMB_suffix_anim
-> MOV_set_multiview_suffix and so on.
- All except one usages of MOV_get_fps (nee IMB_anim_get_fps) were
ultimately just converting returned value into a float. So make
MOV_get_fps just return that directly. For the (exactly just one)
place that needs numerator and denominator, have
MOV_get_fps_num_denom.
- Code comments on the public header functions.
- Removed never-used code paths inside movie timecode proxy building
file.
It might be easier to review each commit separately.
Pull Request: https://projects.blender.org/blender/blender/pulls/132145
Previously, code related to reading/writing movie files via ffmpeg was
scattered around: some under blenkernel, some directly in generic
imbuf headers, some under intern/ffmpeg. Some of the files were named
with not exactly clear names. Some parts not directly related to movies
were including ffmpeg headers directly (rna_scene.cc).
What is in this PR:
Movie and ffmpeg related code is now under imbuf/movie:
- IMB_anim.hh: movie reading, proxy querying, various utility functions.
- IMB_movie_enums.hh: simple enum definitions,
- IMB_movie_write.hh: movie writing functions.
- intern: actual implementation and private headers.
- ffmpeg_compat.h: various ffmpeg version difference handling
utilities,
- ffmpeg_swscale.hh/cc: scaling and format conversion utilities
for ffmpeg libswscale,
- ffmpeg_util.hh/cc: misc utilities related to ffmpeg,
- movie_proxy_indexer.hh/cc: proxies and timecode indexing for movies,
- movie_read.hh/cc: decoding of movies into images,
- movie_write.cc: encoding of images into movies.
- tests: basic ffmpeg library unit tests that previously
lived under intern/ffmpeg.
Interface changes (at C++ level, no Python API changes):
- Mostly just movie related functions that were BKE_ previously, are now IMB_.
- I did one large-ish change though, and that is to remove bMovieHandle
struct that had pointers to several functions. Now that is
IMB_movie_write_begin, IMB_movie_write_append, IMB_movie_write_end
functions using a single opaque struct handle. As a result, usages
of that in pipeline.cc and render_opengl.cc have changed.
Pull Request: https://projects.blender.org/blender/blender/pulls/132074
With exception of Sequencer, everywhere else in Blender it is assumed
that float images are in linear color space.
Movie files with more than 8 bit/channel precision are now loaded into
float images (since 39c4c7cf3f), but the "load and use them as an
image" code path was not making sure that they are converted to linear
color space.
Pull Request: https://projects.blender.org/blender/blender/pulls/131141
Videos files recorded on most phones were coming in sideways; files
recorded on some laptop cameras were coming in upside down. In both
cases the display_matrix metadata of the video stream was not applied.
This required changing internal container format of movie proxies to
be MP4 instead of AVI, since AVI does not support rotation metadata.
File extension kept at ".avi" to not disturb existing expectations.
It might be better to "bake" rotation into proxies while building them,
but right now we do not have a trivial way of doing that.
Note that this is a potential behavior change; if someone
already had manually rotated video strips, they would have
to undo that rotation now.
Pull Request: https://projects.blender.org/blender/blender/pulls/130455
Implements #130836:
interpolate_*_wrapmode_fl take InterpWrapMode wrap_u and wrap_v arguments.
U and V coordinate axes can have different wrap modes: clamp/extend,
border/zero, wrap/repeat.
Note that this removes inconsistency where cubic interpolation was
returning zero for samples completely outside the image, but all other
functions were not, and the behavior was not matching the function
documentation either.
Use the new functions in the new compositor CPU backend.
Possible performance impact for other places (e.g. VSE): measured on
4K resolution, transformed (scaled and rotated) 4K EXR image:
- Nearest filter: no change,
- Bilinear filter: no change,
- Cubic BSpline filter: slight performance decrease, IMB_transform
19.5 -> 20.7 ms (Ryzen 5950X, VS2022). Feels acceptable.
Pull Request: https://projects.blender.org/blender/blender/pulls/130893
This is a more invasive change to the timecode indexer because ffmpeg removed the ability to to get the `pkt_pos` (byte location of a frame packet):
27f8c9b27b
As stated there, they don't think that using it too seek is any better than using the dts or the pts position of the packets.
While you can still seek with the byte position, you need to do quite a bit of extra work to get this information from ffmpeg.
As we are only using it for the indexer and only if the fileformat was quite old (mpegts etc) I thought that we could instead simplify this and always seek by the pts timestamp instead.
Pull Request: https://projects.blender.org/blender/blender/pulls/130444
Tell ffmpeg swscale to do accurate YUV->RGB conversion, instead of
slightly faster but not really accurate one. Fixes banding and some
color shifts in video files, particularly in dark regions.
The accurate conversion is a bit slower though, on 4K resolution video,
time taken to convert video frame from YUV to RGB:
- x64 (Ryzen 5950X): 2.3ms -> 3.7ms
- arm64 (M1 Max): 0.6ms -> 2.9ms
My take is that paying 1-2ms per 4K video playback is acceptable
since the result is obviously "more correct" and matches what VLC/ffplay
produces.
From what I can tell, "accurate conversion" turns off some dedicated
assembly code paths within ffmpeg. Maybe someday ffmpeg would get
accurate and assembly-optimized routines for that.
With more accurate decoding, we can now lower the expected render
test threshold again, since x64 & arm64 decoding is much closer now.
Comparison screenshots in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/130383
Videos written out of blender were not explicitly indicating the color
metadata (color range, primaries, TRC, color space). While some video
players assume BT.709, some others assume BT.601, leading to inconsistent
look.
Explicitly indicate BT.709 and limited ("mpeg") YUV range for video
codecs that are YUV based (i.e. final image is non-RGB(A)). Overall now
the code does the same as these command line ffmpeg parameters:
`-vf scale=out_color_matrix=bt709 -color_primaries 1 -color_trc 1 -colorspace 1`.
This also fixes an issue where video proxies were looking wrong within
Blender, if the source video file contained colorspace metadata.
Images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/130021
NOTE: This also required some changes to Cycles code itself, who is now
directly including `BKE_image.hh` instead of declaring a few prototypes
of these functions in its `blender/utils.h` header (due to C++ functions
names mangling, this was not working anymore).
Pull Request: https://projects.blender.org/blender/blender/pulls/130174
Part of overall #118493 task: video input/output support at 10 and 12
bit/component formats. (note: this is still LDR videos just at higher
precision; there's no HDR handling (yet)).
Movie reading/playback: when movie file pixel format has >8 bit
components, decode those into a floating point ImBuf result. Previously
all movies were decoded into 8 bit/channel ImBufs, so 10- and 12-bit
movie pixel colors were getting quantized.
Movie output: when ffmpeg video with suitable codec is selected,
there's a color depth setting under Encoding block. Currently that is:
- 10 bit option for H.264, H.265, AV1 (VP9 could do 10 bit in theory too,
but ffmpeg that is built in Blender does not have that compiled in)
- 12 bit option for H.265, AV1
When "lossless" is picked, then similar to how for regular 8-bit video
it switches from YUV 4:2:0 to 4:4:4, this also switches to 4:4:4
10- or 12-bit variant.
Pull Request: https://projects.blender.org/blender/blender/pulls/129298
For C/C++ doc-strings should be located in headers,
move function comments into the headers, in some cases merging
with existing doc-strings, in other cases, moving implementation
notes into the function body.
Regression since a904db3ee7 ("skip no-op colorspace transforms for
float images"), the newly added do_display_buffer_apply_no_processor
function did not handle case of both source and destination being
float images. This happens when VSE produces a float image, and
you turn on a Waveform scope.
Pull Request: https://projects.blender.org/blender/blender/pulls/129293
EXR DWAA and DWAB are conceptually similar to lossy JPG compression,
with a tunable file size vs image quality parameter. However, previously
Blender always used the fixed default setting, which is kinda similar
to very high quality (like 97) for JPG.
Internally EXR DWA/DWB quality parameter is inverted scale, i.e. 0 is
best/lossless quality, and increased setting value means decreased
quality. However the rest of Blender UI uses 1-100 JPG-like quality
scale, where values above 90 are "visually lossless", 100 is lossless,
and going below something like 50 would be visually quite lossy. So map
that to internal DWA setting:
- blender 100 -> DWA 0
- blender 97 -> DWA 45
The rest is linear relation based on those two points.
Pull Request: https://projects.blender.org/blender/blender/pulls/128790
IMB_scale modifies the input image. But some places in code needed to keep
original input intact, so what they did was a sequence of IMB_dupImBuf+IMB_scale
Add IMB_scale_into_new function and use that in several obvious places:
- movieclip_build_proxy_ibuf
- icon_copy_rect
- seq_proxy_build_frame
Rebuilding proxies for VSE image sequences with 94 4K resolution EXR images
(on Ryzen 5950X/Win10/VS2022): 13.4 -> 10.3 seconds.
Pull Request: https://projects.blender.org/blender/blender/pulls/128752
Previous fix (b17734598d) tried to fix this, but it seems that
depending on CPU and video width, ffmpeg might need at least 64
byte alignment, even if CPU we're running on is only AVX2 (32 byte
alignment). That is because some other parts of ffmpeg code
statically pick 64 or 32 byte alignment internally, depending on whether
AVX512 support is even compiled in (even if CPU might not have it).
I have checked whether this does not negatively affect a platform where
SIMD alignment is always 16 (Mac), and it does not, everything works as
expected.
Pull Request: https://projects.blender.org/blender/blender/pulls/128107
When building proxies at lower than 100% resolution, the video frame
downscaling step was single threaded, as found via #127956.
Make it use the same threaded sws_scale machinery that the usual video
decoding/encoding uses. Video encoding/decoding was only using it for
RGB<->YUV conversions, so source and destination sizes were always matching;
here it needs to have different source and destination sizes though.
Time taken to rebuild 50% proxy for a 4K resolution 1440 frames (1 minute)
long video file, on Ryzen 5950X (Win10/VS2022):
- Blender 4.2: 20.1 sec, CPU usage 30-40%.
- Blender 4.3 main: 13.1 sec (ffmpeg build has been fixed to use SIMD),
CPU usage still 30-40% though.
- This PR: 8.3 sec, CPU usage ~95%.
Pull Request: https://projects.blender.org/blender/blender/pulls/128054
VSE tonemap is 12-15 times faster.
1) multi-threaded image luminance calculation,
2) avoid calling into OpenColorIO per-pixel, instead do that in
larger batches,
3) for float images (which are primary target for tonemapping),
do not do "to linear space" conversion twice.
Applying tonemap on 4K resolution EXR image, on Ryzen 5950X (Win10/VS2022):
- R/D Photoreceptor mode: 405 -> 31 ms
- Rh Simple mode: 388 -> 23 ms
Pull Request: https://projects.blender.org/blender/blender/pulls/127467
Looks like ffmpeg AVFrame width/height/format for the deinterlaced frame
was never initialized. That was not a problem until starting with 4.1
the colorspace conversion and upside down flip was started to be
multi-threaded, which accessed the frame width/height.
Also, the memory storage for the deinterlaced frame was never freed
either; fix that too.
Pull Request: https://projects.blender.org/blender/blender/pulls/127689
