Replace (only two remaining) usages of C-style IMB_processor_apply_threaded
with just threading::parallel_for which is much easier to use in C++ without
intermediate structs.
IMB_display_buffer_acquire got faster as a result -- parallel for has lower
overhead compared to the task pool approach that the previous
implementation was using. While at it, noticed that
IMB_display_buffer_acquire was clearing just-allocated memory, immediately
before overwriting it. So that is now gone too.
IMB_display_buffer_acquire time during playback of 4K resolution float
content in VSE (Ryzen 5950X, Windows): 10.7ms -> 7.7ms
Pull Request: https://projects.blender.org/blender/blender/pulls/135269
IMB_alpha_under_color_[byte/float] functions are used when preparing
the rendered image for image/movie output with RGB channels (i.e. no
transparency). They were single threaded before, multi-thread them.
Time taken by them on 4K resolution image (mix of various transparency
values in source), on Ryzen 5950X/Windows:
- IMB_alpha_under_color_byte: 10.1ms -> 1.9ms
- IMB_alpha_under_color_float: 14.6ms -> 8.8ms (smaller speedup since
it becomes memory bandwidth limited)
Pull Request: https://projects.blender.org/blender/blender/pulls/135258
OpenEXR DWA compression in Blender is derived from a more user-friendly
quality slider which has an intuitive range 0 .. 100.
Initially the mapping was done so that the visually lossless JPEG
quality of 97 was mapped to the default DWA compression 45. A point was
made that we should make it so default quality is mapped to the default
compression, following the intent of DWA for rendering and compositing
the main target.
This change adjusts the mapping so that quality of 90 is mapped to DWA
compression 45.
This change relies on the library update to fully utilize the DWA
compression #135037.
This change leads to the difference in the way proxies of EXR images
are generated:
```
DWA compression Size (bytes)
Before the change 750 175,208,243
After the change 225 77,838,827
```
It is worth noting that the DWA compression seemed to be ignored in
the 4.4 branch before this change (this is what the original report is
about, a bit indirectly).
This is measured on the Fabrik Eingang footage converted to EXR. The
absolute value is ptobably not that important, it just shows the
reduction in size. This also leads to a lower quality of the proxy
image, but it is not worse than an actual JPEG proxy: the quality is
set to rather low 50 for the strip proxies.
Ref #134802
Pull Request: https://projects.blender.org/blender/blender/pulls/135103
Float->byte rendered image dithering uses triangle noise algorithm. Keep
the algorithm the same, just make some improvements and fix some issues:
1) The hash function for noise was using "trig" hash from "On generating
random numbers" (Rey 1998), but that is not a great quality hash, plus it
can produce very different results between CPUs/GPUs. Replace it with
"iqint3" (recommended by "Hash Functions for GPU Rendering", JCGT 2020),
which is same performance on GPU, faster on CPU, and much better quality.
This is the same hash as Cycles already uses elsewhere. Also it is purely
integer based, so exactly the same results on all platforms.
2) For the above point, replace `dither_random_value` to take integer
pixel coordinates and adjust calling code accordingly. Some previous
callers were (accidentally?) passing integer coordinates already. Other
places actually get a tiny bit simpler, since they now no longer need an
extra multiplication.
3) The CPU dithering path was wrongly introducing bias, i.e. making the
image lighter. The CPU path also needs dither noise to be in [-1..+1]
range (not [-0.5..+1.5]!) just like GPU path does, since the later
float->byte conversion already does rounding.
4) The CPU dithering path was using thread-slice-local Y coordinate,
meaning the dithering pattern was repeating vertically. The more CPU cores
you use, the worse the repetition.
5) Change the way that uniform noise is converted to triangle noise.
Previous implementation was based on one shadertoy from 2015, change it
to another shadertoy from 2020. The new one fixes issues with the old way,
and it just works on the CPU too, so now both CPU and GPU code paths are
exactly the same.
6) Cleanup: remove DitherContext, just a single float is enough
Performance and image comparisons in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/135224
Looks like "divers" comes from ancient times, Dutch word meaning "misc".
But by now, everything in that file is about conversion between different
pixel data types.
Pull Request: https://projects.blender.org/blender/blender/pulls/135165
There's no point in having non-threaded image color space conversion functions.
So merge the threaded and non-threaded functions and clarify names while at it:
- IMB_colormanagement_transform & IMB_colormanagement_transform_threaded
-> IMB_colormanagement_transform_float
- IMB_colormanagement_transform_byte & IMB_colormanagement_transform_byte_threaded
-> IMB_colormanagement_transform_byte
- IMB_colormanagement_transform_from_byte & IMB_colormanagement_transform_from_byte_threaded
-> IMB_colormanagement_transform_byte_to_float
These places were doing single-threaded colorspace conversion previously, and
thus now are potentially faster:
- IMB_rect_from_float (used in many places)
- EXR image "save as render" saving (image_exr_from_scene_linear_to_output)
- Object baking (write_internal_bake_pixels, write_external_bake_pixels)
- General image saving, clipboard copy, movie preparation
(IMB_colormanagement_imbuf_for_write)
- Linear conversion when reading HDR images/movies
(colormanage_imbuf_make_linear)
- EXR multi-layer conversion (render_result_new_from_exr)
For one case I benchmarked, which is to render out a 2D stabilized 10 bit input
movie clip out of VSE, the total render time went from 49sec down to 44sec
(Ryzen 5950X), one of the single-threaded parts was the colorspace conversion
in the movieclip code.
Pull Request: https://projects.blender.org/blender/blender/pulls/135155
Speedup IMB_rotate_orthogonal (used for example in auto-rotating
videos that were shot sideways on a phone) by: 1) not copying previous
pixel values into new result, only for them to be immediately
overwritten by rotated pixels, and 2) using multi-threading.
Performing rotation of 1920x1080 resolution HDR (float) video frame
goes from 20ms down to 5ms (Ryzen 5950X, Windows)
Pull Request: https://projects.blender.org/blender/blender/pulls/135158
The File Output node crashes when saving a 16-bit vector image in an
RGBA image. That's because the OIIO writer assumes 4-channel buffer
while the buffer provided by the node is only 3-channel. To fix this,
the OIIO writer is extended to support all possible combination of
source and target channels.
Pull Request: https://projects.blender.org/blender/blender/pulls/134789
All 2D vectors related to image transform code were changed to float2.
Previously, it was decided, that 4x4 matrix should be used for 2D
affine transform, but this is changed to 3x3 now.
Texture painting code did rely on `IMB_transform` with 4x4 matrix.
To avoid large changes, I have added function
`BLI_rctf_transform_calc_m3_pivot_min`.
Main motivation is cleaner code - ease of use of c++ API, and avoiding
returning values by arguments.
Pull Request: https://projects.blender.org/blender/blender/pulls/133692
Since one user-defined conversion operator is allowed during implicit conversion,
and after this conversion here is a constructor which can accept result
of conversion, there was a backdoor for a vector types to up-cast their
dimensions via cast to pointer type of a component of a vector. Since it was
implicit and non-intentional it led to buffer overflows.
Pull Request: https://projects.blender.org/blender/blender/pulls/132927
* Ensure valid bit depth is set along with file type
* Guard against invalid inputs in stereo imbuf creation
* Remove some unused code
Thanks Yiming Wu for finding the cause.
Pull Request: https://projects.blender.org/blender/blender/pulls/133499
The rest of blender does handle multi-layer EXR images, using the
"combined" or RGBA/RGB layers when the visual result is needed. Make
VSE do the same.
While fixing this, I found several issues in other not well tested code
and had to fix them:
- IMB_buffer_float_from_float_threaded was wrongly using source channels
as destination channels, producing garbage result.
- IMB_scale_into_new was not assigning channels to destination image.
Pull Request: https://projects.blender.org/blender/blender/pulls/132790
When using clangd or running clang-tidy on headers there are
currently many errors. These are noisy in IDEs, make auto fixes
impossible, and break features like code completion, refactoring
and navigation.
This makes source/blender headers work by themselves, which is
generally the goal anyway. But #includes and forward declarations
were often incomplete.
* Add #includes and forward declarations
* Add IWYU pragma: export in a few places
* Remove some unused #includes (but there are many more)
* Tweak ShaderCreateInfo macros to work better with clangd
Some types of headers still have errors, these could be fixed or
worked around with more investigation. Mostly preprocessor
template headers like NOD_static_types.h.
Note that that disabling WITH_UNITY_BUILD is required for clangd to
work properly, otherwise compile_commands.json does not contain
the information for the relevant source files.
For more details see the developer docs:
https://developer.blender.org/docs/handbook/tooling/clangd/
Pull Request: https://projects.blender.org/blender/blender/pulls/132608
This renames the struct `Sequence` to `Strip`.
While the motivation for this partially comes from
the "Sequence Design" #131329, it seems like this
is a good refactor whether the design gets implemented
or not.
The `Sequence` represents what users see as strips in the
VSE. Many places in the code already refere to a `Sequence`
as "strip". It's the C-style "base class" of all strip types.
This also renames the python RNA type `bpy.types.Sequence`
to `bpy.types.Strip` which means that this technically breaks
the python API.
Pull Request: https://projects.blender.org/blender/blender/pulls/132179
This renames the `Strip` struct to `StripData` and also renames
the `Sequence::strip` member to `Sequence::data`.
This is a first step towards naming the `Sequence` struct to `Strip`.
Pull Request: https://projects.blender.org/blender/blender/pulls/132165
- All movie related public headers now have MOV_ prefix instead of
IMB_movie_.
- All movie related public functions now have MOV_ prefix as well,
instead of IMB_movie_ or IMB_anim_.
- IMB_anim.hh -> MOV_read.hh (also ImBufAnim -> MovieReader), and
various utility functions not related to playback were split off
into MOV_util.hh.
- Other function name tweaks for clarity, e.g. IMB_suffix_anim
-> MOV_set_multiview_suffix and so on.
- All except one usages of MOV_get_fps (nee IMB_anim_get_fps) were
ultimately just converting returned value into a float. So make
MOV_get_fps just return that directly. For the (exactly just one)
place that needs numerator and denominator, have
MOV_get_fps_num_denom.
- Code comments on the public header functions.
- Removed never-used code paths inside movie timecode proxy building
file.
It might be easier to review each commit separately.
Pull Request: https://projects.blender.org/blender/blender/pulls/132145
Previously, code related to reading/writing movie files via ffmpeg was
scattered around: some under blenkernel, some directly in generic
imbuf headers, some under intern/ffmpeg. Some of the files were named
with not exactly clear names. Some parts not directly related to movies
were including ffmpeg headers directly (rna_scene.cc).
What is in this PR:
Movie and ffmpeg related code is now under imbuf/movie:
- IMB_anim.hh: movie reading, proxy querying, various utility functions.
- IMB_movie_enums.hh: simple enum definitions,
- IMB_movie_write.hh: movie writing functions.
- intern: actual implementation and private headers.
- ffmpeg_compat.h: various ffmpeg version difference handling
utilities,
- ffmpeg_swscale.hh/cc: scaling and format conversion utilities
for ffmpeg libswscale,
- ffmpeg_util.hh/cc: misc utilities related to ffmpeg,
- movie_proxy_indexer.hh/cc: proxies and timecode indexing for movies,
- movie_read.hh/cc: decoding of movies into images,
- movie_write.cc: encoding of images into movies.
- tests: basic ffmpeg library unit tests that previously
lived under intern/ffmpeg.
Interface changes (at C++ level, no Python API changes):
- Mostly just movie related functions that were BKE_ previously, are now IMB_.
- I did one large-ish change though, and that is to remove bMovieHandle
struct that had pointers to several functions. Now that is
IMB_movie_write_begin, IMB_movie_write_append, IMB_movie_write_end
functions using a single opaque struct handle. As a result, usages
of that in pipeline.cc and render_opengl.cc have changed.
Pull Request: https://projects.blender.org/blender/blender/pulls/132074
With exception of Sequencer, everywhere else in Blender it is assumed
that float images are in linear color space.
Movie files with more than 8 bit/channel precision are now loaded into
float images (since 39c4c7cf3f), but the "load and use them as an
image" code path was not making sure that they are converted to linear
color space.
Pull Request: https://projects.blender.org/blender/blender/pulls/131141
Videos files recorded on most phones were coming in sideways; files
recorded on some laptop cameras were coming in upside down. In both
cases the display_matrix metadata of the video stream was not applied.
This required changing internal container format of movie proxies to
be MP4 instead of AVI, since AVI does not support rotation metadata.
File extension kept at ".avi" to not disturb existing expectations.
It might be better to "bake" rotation into proxies while building them,
but right now we do not have a trivial way of doing that.
Note that this is a potential behavior change; if someone
already had manually rotated video strips, they would have
to undo that rotation now.
Pull Request: https://projects.blender.org/blender/blender/pulls/130455
Implements #130836:
interpolate_*_wrapmode_fl take InterpWrapMode wrap_u and wrap_v arguments.
U and V coordinate axes can have different wrap modes: clamp/extend,
border/zero, wrap/repeat.
Note that this removes inconsistency where cubic interpolation was
returning zero for samples completely outside the image, but all other
functions were not, and the behavior was not matching the function
documentation either.
Use the new functions in the new compositor CPU backend.
Possible performance impact for other places (e.g. VSE): measured on
4K resolution, transformed (scaled and rotated) 4K EXR image:
- Nearest filter: no change,
- Bilinear filter: no change,
- Cubic BSpline filter: slight performance decrease, IMB_transform
19.5 -> 20.7 ms (Ryzen 5950X, VS2022). Feels acceptable.
Pull Request: https://projects.blender.org/blender/blender/pulls/130893
This is a more invasive change to the timecode indexer because ffmpeg removed the ability to to get the `pkt_pos` (byte location of a frame packet):
27f8c9b27b
As stated there, they don't think that using it too seek is any better than using the dts or the pts position of the packets.
While you can still seek with the byte position, you need to do quite a bit of extra work to get this information from ffmpeg.
As we are only using it for the indexer and only if the fileformat was quite old (mpegts etc) I thought that we could instead simplify this and always seek by the pts timestamp instead.
Pull Request: https://projects.blender.org/blender/blender/pulls/130444
Tell ffmpeg swscale to do accurate YUV->RGB conversion, instead of
slightly faster but not really accurate one. Fixes banding and some
color shifts in video files, particularly in dark regions.
The accurate conversion is a bit slower though, on 4K resolution video,
time taken to convert video frame from YUV to RGB:
- x64 (Ryzen 5950X): 2.3ms -> 3.7ms
- arm64 (M1 Max): 0.6ms -> 2.9ms
My take is that paying 1-2ms per 4K video playback is acceptable
since the result is obviously "more correct" and matches what VLC/ffplay
produces.
From what I can tell, "accurate conversion" turns off some dedicated
assembly code paths within ffmpeg. Maybe someday ffmpeg would get
accurate and assembly-optimized routines for that.
With more accurate decoding, we can now lower the expected render
test threshold again, since x64 & arm64 decoding is much closer now.
Comparison screenshots in the PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/130383
Videos written out of blender were not explicitly indicating the color
metadata (color range, primaries, TRC, color space). While some video
players assume BT.709, some others assume BT.601, leading to inconsistent
look.
Explicitly indicate BT.709 and limited ("mpeg") YUV range for video
codecs that are YUV based (i.e. final image is non-RGB(A)). Overall now
the code does the same as these command line ffmpeg parameters:
`-vf scale=out_color_matrix=bt709 -color_primaries 1 -color_trc 1 -colorspace 1`.
This also fixes an issue where video proxies were looking wrong within
Blender, if the source video file contained colorspace metadata.
Images in PR.
Pull Request: https://projects.blender.org/blender/blender/pulls/130021
NOTE: This also required some changes to Cycles code itself, who is now
directly including `BKE_image.hh` instead of declaring a few prototypes
of these functions in its `blender/utils.h` header (due to C++ functions
names mangling, this was not working anymore).
Pull Request: https://projects.blender.org/blender/blender/pulls/130174
Part of overall #118493 task: video input/output support at 10 and 12
bit/component formats. (note: this is still LDR videos just at higher
precision; there's no HDR handling (yet)).
Movie reading/playback: when movie file pixel format has >8 bit
components, decode those into a floating point ImBuf result. Previously
all movies were decoded into 8 bit/channel ImBufs, so 10- and 12-bit
movie pixel colors were getting quantized.
Movie output: when ffmpeg video with suitable codec is selected,
there's a color depth setting under Encoding block. Currently that is:
- 10 bit option for H.264, H.265, AV1 (VP9 could do 10 bit in theory too,
but ffmpeg that is built in Blender does not have that compiled in)
- 12 bit option for H.265, AV1
When "lossless" is picked, then similar to how for regular 8-bit video
it switches from YUV 4:2:0 to 4:4:4, this also switches to 4:4:4
10- or 12-bit variant.
Pull Request: https://projects.blender.org/blender/blender/pulls/129298
For C/C++ doc-strings should be located in headers,
move function comments into the headers, in some cases merging
with existing doc-strings, in other cases, moving implementation
notes into the function body.
