GOP size and quality are adjusted for h264 codec.
These new values are based on result of benchmark on 9 random files:
https://docs.google.com/spreadsheets/d/1nOyUGjoVWUyhQ2y2lAd8VtFfyaY1wQNGj1krCCNbk7Y/edit?usp=sharing
Reducing quality to 50 reduces proxy filesize by about 2x on average
and has no significant impact on decoding performance.
Increasing GOP size from 2 to 10 also reduces proxy filesize 2x-3x
while scrubbing is only about 8% slower. It is still around 100FPS
with 1920x1080 media.
This is unfortunately about 50% slower than MJPEG, but this can be
improved with `fastdecode` tune applied to libx264 encoder
Quite surprisingly h264 codec presets had little influence on proxy
building performance as well as proxy filesize. So far it looks that
FFmpeg does initialize encoder in different way then Blender.
This applies mot only for presets but for tune and profile libx264
setting.
Once this issue is resolved, performance of proxies may be optimized
further.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10897
Adds appropriate checks/guards around all the untrusted parameters
which are used for reading from memory.
Validation:
- All the crashing files within the bug have been checked to not causes
crashes any longer>
- A handful of correct .bmp were validated: 3 different files at each
of 1, 4, 8, 24, 32 bpp depth along with a random variety of other 24
bpp files (around 20 in total).
- ~280 million iterations of fuzzing using AFL were completed with 0
crashes. The old code experienced several dozen crashes in first
minutes of running {F8584509}.
Ref D7945
This removes a lot of unnecessary code that is generated by
the compiler automatically.
In very few cases, a defaulted destructor in a .cc file is
still necessary, because of forward declarations in the header.
I removed some defaulted virtual destructors, because they are not
necessary, when the parent class has a virtual destructor already.
Defaulted constructors are only necessary when there is another
constructor, but the class should still be default constructible.
Differential Revision: https://developer.blender.org/D10911
This condition was in contradiction with comment for function
`ffmpeg_generic_seek_workaround()`.
I have noticed, that formats that seeked well used this workaround.
Problem was that I misunderstood code from `av_seek_frame()` - formats
with `read_seek()` function stil don't use generic seeking method.
This is defined in `seek_frame_internal()`
`ffmpeg_generic_seek_workaround()` applied negative offset for seqrched
packet timestamp, but proxies always start from 0 and timestamp can be
negative.
Limit timestamp value to 0, because `av_seek_frame()` doesn't accept
negative timestamps and returns with error. This prevents seeking from
working correctly.
`av_seek_frame()` failed to seek to nearest I-frame. This seems to be
a bug or not implemented feature in FFmpeg. Looks like same issue as
ticket https://trac.ffmpeg.org/ticket/1607 on ffmpeg tracker.
If seeking is done using format specific function (`read_seek2`)
field of `AVInputFormat` is set, `see av_seek_frame()`, use
`av_seek_frame()` function. Otherwise use wrapper that actively searches
for I-frame packet.
Searching is flexible and tries to do minimum amount of work. Currently
it is limited to equivalent of 25 frames, which may not be enough for
some files, but there may be files with no I-frames at all, so it is
best to keep this limit as low as possible. Previously this problem was
masked by preseek, which was hard-coded to 25 frames. This was removed
in rB88604b79b7d1.
If this approach would be unnecessary for some formats, in worst case
file would be seeked 2 times which is very fast, so there will be no
visible impact on performance.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10845
When reading metadata from image files the metadata is trimmed to 1024.
For cryptomatte the metadata can contain json data and should not be
trimmed. Resulting additional checks in the manifest parser for
incomplete json data.
You could argue to add an exception for cryptomatte, but that would
still allows misuse. When the direction of this patch is accepted we
should consider removing `maxlen` from `IDP_AssignString` as it
doesn't seem to be used anywhere else.
Reviewed By: #images_movies, mont29, sergey
Differential Revision: https://developer.blender.org/D10825
Generalize threading settings in proxy building and use them for encoding
and decoding in general. Check codec capabilities, prefer FF_THREAD_FRAME
threading over FF_THREAD_SLICE and automatic thread count over setting it
explicitly.
ffmpeg-codecs man page suggests that threads option is global and used by
codecs, that supports this option. Form some tests I have done, it seems that
`av_dict_set_int(&codec_opts, "threads", BLI_system_thread_count(), 0)`
has same effect as
```
pCodecCtx->thread_count = BLI_system_thread_count();
pCodecCtx->thread_type = FF_THREAD_FRAME;
```
Looking at `ff_frame_thread_encoder_init()` code, these cases are not
equivalent. It is probably safer to leave threading setup on libavcodec than
setting up each codec threading individually.
From what I have read all over the internet, frame multithreading should be
faster than slice multithreading. Slice multithreading is mainly used for low
latency streaming.
When running Blender with --debug-ffmpeg it complains about
`pCodecCtx->thread_count = BLI_system_thread_count()` that using thread count
above 16 is not recommended. Using too many threads can negatively affect image
quality, but I am not sure if this is the case for decoding as well - see
https://streaminglearningcenter.com/blogs/ffmpeg-command-threads-how-it-affects-quality-and-performance.html
This is fine for proxies but may be undesirable for final renders.
Number of threads is limited by image size, because of size of motion vectors,
so if it is possible let libavcodec determine optimal thread count.
Performance difference:
Proxy building: None
Playback speed: 2x better on 1920x1080 sample h264 file
Scrubbing: Hard to quantify, but it's much more responsive
Rendering speed: None on 1920x1080 sample h264 file, there is improvement with codecs that do support FF_THREAD_FRAME for encoding like MPNG
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10791
Split seeking section of `ffmpeg_fetchibuf()` function into multiple
smaller functions.
Conditional statements are moved to own funtions with human readable
names, so code flow is more clear.
To remove one branch of seeking, first frame is now decoded by
scanning, which will do only one iteration. So nothing has technically
changed.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10638
Use h264 codec for output. This codec produces smaller files, can be
multithreaded and decodes even faster than MJPEG.
Quality setting 0-100 corresponds to "Lowest Quality" to
"Perceptually Lossless" in Blender's h264 encoding presets.
All available cores are used for decoding.
Same goes for decoding but only for codecs that supports this
(h264, vp9 seems to support this option out of th box as well).
Other decoders can probably be optimized in similar way, but threaded
encoding provides significant boost already.
I have tested variety of codecs, and all were transcoded properly.
Reviewed By: sergey, fsiddi
Differential Revision: https://developer.blender.org/D10731
Applying negative offset to seek position before scanning doesnn't have
any effect. This change results in 1.5x faster seeking (random frame,
average value) in sample file with 30 frame GOP length.
If I am not mistaken, B frames can have pts that can be less than
pts of I frame that must be decoded. Even in this case though, B frame
packet will be stored after that I frame.
In addition, preseek value is de facto hardcoded so seeking would fail
if it could. This can be hard to spot though.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10529
Removal of 'camera frame' around blend file thumbnail images.
Differential Revision: https://developer.blender.org/D10490
Reviewed by Brecht Van Lommel
Following code from D8627 this patch corrects multi threaded processing
of proxies, where a 60 sec proxy generation drops to 35 sec.
Differential Revision: https://developer.blender.org/D8659
Allow use all system threads for frame encoding/decoding. This is very
straightforward: the value of zero basically disables threading.
Change threading policy to slice when decoding frames. The reason for
this is because decoding happens frame-by-frame, so inter-frame threading
policy will not bring any speedup.
The change for threading policy to slice is less obvious and is based on
benchmark of the demo files from T78986. This gives best performance so
far.
Rendering the following file went down from 190sec down to 160sec.
https://storage.googleapis.com/institute-storage/vse_simplified_example.zip
This change makes both reading and writing faster. The animation render
is just easiest to get actual time metrics.
Differential Revision: https://developer.blender.org/D8627
Add a boundary check, avoiding access past actual data.
Ideally would need to report error to the user somehow,
but it doesn't seem to be easy to do.
This is a minimal safe patch. The proper complete fix is
being worked on by Jesse.
Differential Revision: https://developer.blender.org/D10357
Ref T84819
Build System
============
This is an API breaking new version, and the updated code only builds with
OpenColorIO 2.0 and later. Adding backwards compatibility was too complicated.
* Tinyxml was replaced with Expat, adding a new dependency.
* Yaml-cpp is now built as a dependency on Unix, as was already done on Windows.
* Removed currently unused LCMS code.
* Pystring remains built as part of OCIO itself, since it has no good build system.
* Linux and macOS check for the OpenColorIO verison, and disable it if too old.
Ref D10270
Processors and Transforms
=========================
CPU processors now need to be created to do CPU processing. These are cached
internally, but the cache lookup is not fast enough to execute per pixel or
texture sample, so for performance these are now also exposed in the C API.
The C API for transforms will no longer be needed afer all changes, so remove
it to simplify the API and fallback implementation.
Ref D10271
Display Transforms
==================
Needs a bit more manual work constructing the transform. LegacyViewingPipeline
could also have been used, but isn't really any simpler and since it's legacy
we better not rely on it.
We moved more logic into the opencolorio module, to simplify the API. There is
no need to wrap a dozen functions just to be able to do this in C rather than C++.
It's also tightly coupled to the GPU shader logic, and so should be in the same
module.
Ref D10271
GPU Display Shader
==================
To avoid baking exposure and gamma into the GLSL shader and requiring slow
recompiles when tweaking, we manually apply them in the shader. This leads
to some logic duplicaton between the CPU and GPU display processor, but it
seems unavoidable.
Caching was also changed. Previously this was done both on the imbuf and
opencolorio module levels. Now it's all done in the opencolorio module by
simply matching color space names. We no longer use cacheIDs from OpenColorIO
since computing them is expensive, and they are unlikely to match now that
more is baked into the shader code.
Shaders can now use multiple 2D textures, 3D textures and uniforms, rather
than a single 3D texture. So allocating and binding those adds some code.
Color space conversions for blending with overlays is now hardcoded in the
shader. This was using harcoded numbers anyway, if this every becomes a
general OpenColorIO transform it can be changed, but for now there is no
point to add code complexity.
Ref D10273
CIE XYZ
=======
We need standard CIE XYZ values for rendering effects like blackbody emission.
The relation to the scene linear role is based on OpenColorIO configuration.
In OpenColorIO 2.0 configs roles can no longer have the same name as color
spaces, which means our XYZ role and colorspace in the configuration give an
error.
Instead use the new standard aces_interchange role, which relates scene linear
to a known scene referred color space. Compatibility with the old XYZ role is
preserved, if the configuration file has no conflicting names.
Also includes a non-functional change to the configuraton file to use an
XYZ-to-ACES matrix instead of REC709-to-ACES, makes debugging a little easier
since the matrix is the same one we have in the code now and that is also
found easily in the ACES specs.
Ref D10274
This patch will show textures in the image editor with the maximum
available resolution determined by the GPU Hardware/Driver.
Currently the size is limited by the user preference texture size limit.
An image user can set the `IMA_SHOW_MAX_RESOLUTION` flag to request
gpu textures in the max supported resolution. When this flag isn't
set the gpu texture is limited by the user preference setting.
When the gl resolution limit is disabled the GPU texture is always
created for the max supported resolution.
Reviewed By: Clément Foucault
Maniphest Tasks: T81206
Differential Revision: https://developer.blender.org/D9160
This patch will show textures in the image editor with the maximum
available resolution determined by the GPU Hardware/Driver.
Currently the size is limited by the user preference texture size limit.
An image user can set the `IMA_SHOW_MAX_RESOLUTION` flag to request
gpu textures in the max supported resolution. When this flag isn't
set the gpu texture is limited by the user preference setting.
When the gl resolution limit is disabled the GPU texture is always
created for the max supported resolution.
Reviewed By: Clément Foucault
Maniphest Tasks: T81206
Differential Revision: https://developer.blender.org/D9160
Replace `long long` with an explicit `int64_t`. This is also what is
used in the FFmpeg headers.
Fixes clang diagnostics warning about wrong format used in the log.
Should be no functional changes.
For jpeg, an image.resolution was always based on the default 72dpi, now
read the pixel density from the jpeg_decompress_struct, convert
according to unit and store in IMBuf's ppm.
Not 100% sure of all implications tbh., files I have checked seem to work
as expected now in the context of the report.
Maniphest Tasks: T84661
Differential Revision: https://developer.blender.org/D10166
Instead of submitting tons of tiny IO syscalls, we can speed things up
significantly by `mmap`ing the .blend file into virtual memory and directly
accessing it.
In my local testing, this speeds up loading the Dweebs file with all its
linked files from 19sec to 10sec (on Linux).
As far as I can see, this should be supported on Linux, OSX and BSD.
For Windows, a second code path uses `CreateFileMapping` and
`MapViewOfFile` to achieve the same result.
Reviewed By: mont29, brecht
Differential Revision: https://developer.blender.org/D8246
Clamp value to the -HALF_MAX .. HALF_MAX.
The non-clamped values were causing NaN and inf values saved to
the file, which was the root cause of glare node giving unexpected
result.
The nan/inf on overflow is something mentioned in the half data
type in OpenEXR header.
Differential Revision: https://developer.blender.org/D10105
Without this, additional items could be added in the future
which wouldn't be included in the check.
Move the check out of the loop since this is such an unlikely
situation that checking every iteration isn't needed.
Also remove redundant casts.
Handle return value of `fread()` by printing an error and closing the
file when it cannot be read from. Not only is error handing a good idea,
it also prevents GCC from warning that the return value of `fread()`
should not be ignored:
```
.../blender/source/blender/imbuf/intern/indexer.c: In function ‘IMB_indexer_open’:
.../blender/source/blender/imbuf/intern/indexer.c:201:5: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
201 | fread(&idx->entries[i].frameno, sizeof(int), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../blender/source/blender/imbuf/intern/indexer.c:202:5: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
202 | fread(&idx->entries[i].seek_pos, sizeof(unsigned long long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../blender/source/blender/imbuf/intern/indexer.c:203:5: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
203 | fread(&idx->entries[i].seek_pos_dts, sizeof(unsigned long long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../blender/source/blender/imbuf/intern/indexer.c:204:5: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
204 | fread(&idx->entries[i].pts, sizeof(unsigned long long), 1, fp);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Differential Revision: https://developer.blender.org/D9916
Reviewed by: campbellbarton
