When writing .blend files, Blender traditionally wrote raw runtime-pointers into
the file. Since these pointers are different whenever Blender is restarted or
the file is loaded again, the written .blend file will be very different every
time. The file always changing is a problem with tools that use change-detection
on .blend files such as:
* Change detection during undo in Blender. When a serialized data-block is the
same in two consecutive undo steps, it's known that the data didn't change and
the data-block does not have to be reloaded and can potentially skip depsgraph
evaluation. Also see #141262.
* BAT: https://projects.blender.org/studio/flamenco/issues/104437
* Generally using .blend files in version control. The diffs can be smaller when
pointers aren't changing all the time and have a lower memory footprint.
This PR makes pointers in .blend files across multiples saves much more
consistent, improving the situation for the cases above. Although there is still
other data that changes on almost every save currently; that needs to be
addressed separately.
The basic design for pointer stability in blend files stable pointers, described
in #127706, is fairly straight forward. This patch implements a slightly
modified variant of that design. When reading .blend files, Blender already does
not care if the stored pointer values are actual pointer values, or just some
arbitrary identifiers. Therefore, we mainly just have to change the write-file
code. This also implies that this change is fully forward and backward
compatible.
The main non-obvious aspect of this patch is how to actually do the remapping of
runtime pointers to stable identifiers. In theory, having a single integer that
increments for every newly detected pointer works. But in practice that leads to
many changes in the .blend file because one pointer is added or removed
somewhere, all subsequent pointers will be different too. So some kind of
scoping is required to make sure that one small change does not affect
everything else.
This PR starts a new scope pointer identifier scope whenever a new ID data-block
starts. At first I thought it would be good to have separate maps for id-local
and global pointers. However, that's tricky currently, because at write-time, we
don't always have enough information to know if a specific pointer is local or
global. I worked on #146136 to improve the situation but the problem is bigger
than that since we also have various void pointers in DNA structs. Fortunately,
the solution implemented now also works fine with a single global map.
Implicit sharing in undo steps also had to be changed slightly to work with the
stable address identifiers instead of raw pointers.
There's also new code to find all pointers in DNA structs in the first place.
This is done once when writing starts. Then whenever a struct is written, a copy
is made, all pointers are replaced and the modified struct is written to the
.blend file. There is an optimization for the case when a struct does not
contain any pointers because then the copy can be skipped.
For checking the diff between two saved .blend files, I recommend enabling
`GENERATE_DEBUG_BLEND_FILE` and then diffing the text version of the .blend
files.
Co-authored-by: Hans Goudey <hans@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/127729
cadb3fe5c5