Adding better support for drawing huge images in the image/uv editor. Also solved tearing artifacts.
The approach is that for each image/uv editor a screen space gpu texture is created that only contains
the visible pixels. When zooming or panning the gpu texture is rebuild.
Although the solution isn't memory intensive other parts of blender memory usage scales together with
the image size.
* Due to complexity we didn't implement partial updates when drawing images tiled (wrap repeat).
This could be added, but is complicated as a change in the source could mean many different
changes on the GPU texture. The work around for now is to tag all gpu textures to be dirty when
changes are detected.
Original plan was to have 4 screen space images to support panning without gpu texture creation.
For now we don't see the need to implement it as the solution is already fast. Especially when
GPU memory is shared with CPU ram.
Reviewed By: fclem
Maniphest Tasks: T92525, T92903
Differential Revision: https://developer.blender.org/D13424
Assert when "//" prefixed relative paths are passed to
BLI_path_cmp_normalized as this can't be expanded
and it's possible the paths come from different blend files.
Changes to recent addition: c85c52f2ce.
Having both BLI_paths_equal and BLI_path_cmp made it ambiguous
which should be used, as `BLI_paths_equal` wasn't the equivalent to
`BLI_path_cmp(..) == 0` as it is for string equals macro `STREQ(..)`.
It's also a more specialized function which is not used for path
comparison throughout Blender's internal path handling logic.
Instead rename this `BLI_path_cmp_normalized` and return the result of
`BLI_path_cmp` to make it clear paths are modified before comparison.
Also add comments about the conventions for Blender's path comparison
as well as a possible equivalent to Python's `os.path.samefile`
for checking if two paths point to the same location on the file-system.
Add `USD Preview Surface From Nodes` export option, to convert a
Principled BSDF material node network to an approximate USD Preview
Surface shader representation. If this option is disabled, the original
material export behavior is maintained, where viewport setting are saved
to the Preview Surface shader.
Also added the following options for texture export.
- `Export Textures`: If converting Preview Surface, export textures
referenced by shader nodes to a 'textures' directory which is a
sibling of the USD file.
- `Overwrite Textures`: Allow overwriting existing texture files when
exporting textures (this option is off by default).
- `Relative Texture Paths`: Make texture asset paths relative to the
USD.
The entry point for the new functionality is
`create_usd_preview_surface_material()`, called from
`USDAbstractWriter::ensure_usd_material()`. The material conversion
currently handles a small subset of Blender shading nodes,
`BSDF_DIFFUSE`, `BSDF_PRINCIPLED`, `TEX_IMAGE` and `UVMAP`.
Texture export is handled by copying texture files from their original
location to a `textures` folder in the same directory as the USD.
In-memory and packed textures are saved directly to the textures folder.
This patch is based, in part, on code in Tangent Animation's USD
exporter branch.
Reviewed By: sybren, HooglyBoogly
Differential Revision: https://developer.blender.org/D13647
As described in T91186, this commit moves mesh vertex normals into a
contiguous array of float vectors in a custom data layer, how face
normals are currently stored.
The main interface is documented in `BKE_mesh.h`. Vertex and face
normals are now calculated on-demand and cached, retrieved with an
"ensure" function. Since the logical state of a mesh is now "has
normals when necessary", they can be retrieved from a `const` mesh.
The goal is to use on-demand calculation for all derived data, but
leave room for eager calculation for performance purposes (modifier
evaluation is threaded, but viewport data generation is not).
**Benefits**
This moves us closer to a SoA approach rather than the current AoS
paradigm. Accessing a contiguous `float3` is much more efficient than
retrieving data from a larger struct. The memory requirements for
accessing only normals or vertex locations are smaller, and at the
cost of more memory usage for just normals, they now don't have to
be converted between float and short, which also simplifies code
In the future, the remaining items can be removed from `MVert`,
leaving only `float3`, which has similar benefits (see T93602).
Removing the combination of derived and original data makes it
conceptually simpler to only calculate normals when necessary.
This is especially important now that we have more opportunities
for temporary meshes in geometry nodes.
**Performance**
In addition to the theoretical future performance improvements by
making `MVert == float3`, I've done some basic performance testing
on this patch directly. The data is fairly rough, but it gives an idea
about where things stand generally.
- Mesh line primitive 4m Verts: 1.16x faster (36 -> 31 ms),
showing that accessing just `MVert` is now more efficient.
- Spring Splash Screen: 1.03-1.06 -> 1.06-1.11 FPS, a very slight
change that at least shows there is no regression.
- Sprite Fright Snail Smoosh: 3.30-3.40 -> 3.42-3.50 FPS, a small
but observable speedup.
- Set Position Node with Scaled Normal: 1.36x faster (53 -> 39 ms),
shows that using normals in geometry nodes is faster.
- Normal Calculation 1.6m Vert Cube: 1.19x faster (25 -> 21 ms),
shows that calculating normals is slightly faster now.
- File Size of 1.6m Vert Cube: 1.03x smaller (214.7 -> 208.4 MB),
Normals are not saved in files, which can help with large meshes.
As for memory usage, it may be slightly more in some cases, but
I didn't observe any difference in the production files I tested.
**Tests**
Some modifiers and cycles test results need to be updated with this
commit, for two reasons:
- The subdivision surface modifier is not responsible for calculating
normals anymore. In master, the modifier creates different normals
than the result of the `Mesh` normal calculation, so this is a bug
fix.
- There are small differences in the results of some modifiers that
use normals because they are not converted to and from `short`
anymore.
**Future improvements**
- Remove `ModifierTypeInfo::dependsOnNormals`. Code in each modifier
already retrieves normals if they are needed anyway.
- Copy normals as part of a better CoW system for attributes.
- Make more areas use lazy instead of eager normal calculation.
- Remove `BKE_mesh_normals_tag_dirty` in more places since that is
now the default state of a new mesh.
- Possibly apply a similar change to derived face corner normals.
Differential Revision: https://developer.blender.org/D12770
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
This patch implements the vector types (i.e:float2) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the blender::math namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others we
currently don't have (uintX, intX). All these variations were asking
for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector functions
should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the BLI_(float|double|mpq)(2|3|4).hh is a bit of a
let down. Most clases are incomplete, out of sync with each others with
different codestyles, and some functions that should be static are not
(i.e: float3::reflect()).
Upsides:
- Still support .x, .y, .z, .w for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types and
can be restricted to certain types. Also template specialization let us
define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance is
the same.
Downsides:
- Might impact debugability. Though I would arge that the bugs are rarelly
caused by the vector class itself (since the operations are quite trivial)
but by the type conversions.
- Might impact compile time. I did not saw a significant impact since the
usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length. For
instance, one can't call len_squared_v3v3 in math::length_squared() and
call it a day.
- Type cast does not work with the template version of the math:: vector
functions. Meaning you need to manually cast float * and (float *)[3] to
float3 for the function calls.
i.e: math::distance_squared(float3(nearest.co), positions[i]);
- Some parts might loose in readability:
float3::dot(v1.normalized(), v2.normalized())
becoming
math::dot(math::normalize(v1), math::normalize(v2))
But I propose, when appropriate, to use
using namespace blender::math; on function local or file scope to
increase readability. dot(normalize(v1), normalize(v2))
Consideration:
- Include back .length() method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement.
It felt like too much for what we need and would be difficult to
extend / modify to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches delaunay_2d.cc and the intersection code. I would like to
know @Howard Trickey (howardt) opinion on the matter.
- The noexcept on the copy constructor of mpq(2|3) is being removed.
But according to @Jacques Lucke (JacquesLucke) it is not a real problem
for now.
I would like to give a huge thanks to @Jacques Lucke (JacquesLucke) who
helped during this and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: http://developer.blender.org/D13791
It it rather an old experiment now which didn't pay off.
The initial idea was to have main and jobs threads on fast
nodes of TR2 processors. This didn't really work reliably
because in Blender we need to be able to create nested
threads without their affinity set. This is not how some of
OS are creating nested threads, and we don't always have
access to child threads to reset their affinity.
So overall complexity of the initial idea implementation
became too much compared to the performance gain.
Function `blend_color_softlight_float` used math different to compositor and
produced result that had abrupt value changes.
Use math based on modified screen blend mode as compositor does.
This implements the design detailed in T92696 to support virtual
filenames for UDIM textures. Currently, the following 2 substitution
tokens are supported:
| Token | Meaning |
| ----- | ---- |
| <UDIM> | 1001 + u-tile + v-tile * 10 |
| <UVTILE> | Equivalent to u<u-tile + 1>_v<v-tile + 1> |
Example for u-tile of 3 and v-tile of 1:
filename.<UDIM>_ver0023.png --> filename.1014_ver0023.png
filename.<UVTILE>_ver0023.png --> filename.u4_v2_ver0023.png
For image loading, the existing workflow is unchanged. A user can select
one or more image files, belonging to one or more UDIM tile sets, and
have Blender load them all as it does today. Now the <UVTILE> format is
"guessed" just as the <UDIM> format was guessed before.
If guessing fails, the user can simply go into the Image Editor and type
the proper substitution in the filename. Once typing is complete,
Blender will reload the files and correctly fill the tiles. This
workflow is new as attempting to fix the guessing in current versions
did not really work, and the user was often stuck with a confusing
situation.
For image saving, the existing workflow is changed slightly. Currently,
when saving, a user has to be sure to type the filename of the first
tile (e.g. filename.1001.png) to save the entire UDIM set. The number
could differ if they start at a different tile etc. This is confusing.
Now, the user should type a filename containing the appropriate
substitution token. By default Blender will fill in a default name using
the <UDIM> token but the user is free to save out images using <UVTILE>
if they wish.
Differential Revision: https://developer.blender.org/D13057
The cast to size_t leads to a build issue on 32
bit archs. cursor_delim_type_utf8 expects an int
so an additional cast to size_t is not required.
Reported by user frispete on devtalk.
On Windows, encode file paths as UTF-16 before trying to open the file
for reading/writing.
This introduces a new class `blender::fstream`, which wraps
`std::fstream` and provides this UTF-16 encoding. This class should also
be used in other areas, like the Alembic importer/exporter.
Manifest Task: T93960
Reviewed By: JacquesLucke
Differential Revision: https://developer.blender.org/D13633
The RNA setter now ensures that driver variables are uniquely named
(within the scope of the driver).
Versioning code has been added to ensure this uniqueness. The last
variable with the non-unique name retains the original name; this
ensures that the driver will still evaluate to the same value as before
this fix.
This also introduces a new blenlib function `BLI_listbase_from_link()`,
which can be used to find the entire list from any item within the list.
Manifest Task: T94116
Reviewed By: mont29, JacquesLucke
Maniphest Tasks: T94116
Differential Revision: https://developer.blender.org/D13594
Using the `MEM_*` API from C++ code was a bit annoying:
* When converting C to C++ code, one often has to add a type cast on
returned `void *`. That leads to having the same type name three times
in the same line. This patch reduces the amount to two and removes the
`sizeof(...)` from the line.
* The existing alternative of using `OBJECT_GUARDED_NEW` looks a out
of place compared to other allocation methods. Sometimes
`MEM_CXX_CLASS_ALLOC_FUNCS` can be used when structs are defined
in C++ code. It doesn't look great but it's definitely better. The downside
is that it makes the name of the allocation less useful. That's because
the same name is used for all allocations of a type, independend of
where it is allocated.
This patch introduces three new functions: `MEM_new`, `MEM_cnew` and
`MEM_delete`. These cover the majority of use cases (array allocation is
not covered).
The `OBJECT_GUARDED_*` macros are removed because they are not
needed anymore.
Differential Revision: https://developer.blender.org/D13502
Saving with only a filename (from Python) wasn't being prevented,
while it would successfully write the file to the working-directory,
path remapping and setting relative paths wouldn't work afterwards
as `Main.filepath` would have no directory component.
Disallow this since it's a corner case which only ever occurs
when path names without any directories are used from Python,
the overhead of expanding the working-directory for all data saving
operations isn't worthwhile.
The following changes have been made:
- bpy.ops.wm.save_mainfile() without a filepath argument
fails & reports and error when the file hasn't been saved.
Previously it would write to "untitled.blend" and set the
`G.main->filepath` to this as well.
- bpy.ops.wm.save_mainfile(filepath="untitled.blend")
fails & reports and error as the filename has no directory component.
- `BLI_path_is_abs_from_cwd` was added to check if the path would
attempt to expand to the CWD.
This builds off of rBf951aa063f7, adding a weight parameter which can
be used to change the order of items when they have the same match
score. In the future, if string searching gets a C++ API, we could
use an optional parameter for the weight, since it is not used yet.
This will be used for the node link drag search menu (D8286).
Differential Revision: https://developer.blender.org/D13559
Replaces `HOME` environment variable usage for user
directories like in D12802.
Reviewed By: #platform_macos, brecht
Differential Revision: https://developer.blender.org/D13212
Some doc-strings were skipped because of blank-lines between
the doc-string and the symbol and needed to be moved manually.
- Added space below non doc-string comments to make it clear
these aren't comments for the symbols directly below them.
- Use doxy sections for some headers.
Ref T92709
- Added space below non doc-string comments to make it clear
these aren't comments for the symbols directly below them.
- Use doxy sections for some headers.
- Minor improvements to doc-strings.
Ref T92709
The issue here was that after the seek table check, the underlying file wasn't
rewound to the start, so the code that checks for the BLENDER header
immediately reaches EOF and fails.
Since Blender always writes files with a seek table, this bug isn't triggered
by files saved in Blender itself. However, files compressed in external tools
generally don't have a seek table.
Dirty regions when painting are not using rcti. Meaning less
understandable code. Found issue when refactoring the image_gpu partial
update. In a future change gpu partial update API will be using rcti
also what makes the code even cleaner.
Reviewed By: campbellbarton
Differential Revision: https://developer.blender.org/D13260
