Refactoring mesh code, it has become clear that local cleanups and
simplifications are limited by the need to keep a C public API for
mesh functions. This change makes code more obvious and makes further
refactoring much easier.
- Add a new `BKE_mesh.hh` header for a C++ only mesh API
- Introduce a new `blender::bke::mesh` namespace, documented here:
https://wiki.blender.org/wiki/Source/Objects/Mesh#Namespaces
- Move some functions to the new namespace, cleaning up their arguments
- Move code to `Array` and `float3` where necessary to use the new API
- Define existing inline mesh data access functions to the new header
- Keep some C API functions where necessary because of RNA
- Move all C++ files to use the new header, which includes the old one
In the future it may make sense to split up `BKE_mesh.hh` more, but for
now keeping the same name as the existing header keeps things simple.
Pull Request: https://projects.blender.org/blender/blender/pulls/105416
Currently the shade smooth status for mesh faces is stored as part of
`MPoly::flag`. As described in #95967, this moves that information
to a separate boolean attribute. It also flips its status, so the
attribute is now called `sharp_face`, which mirrors the existing
`sharp_edge` attribute. The attribute doesn't need to be allocated
when all faces are smooth. Forward compatibility is kept until
4.0 like the other mesh refactors.
This will reduce memory bandwidth requirements for some operations,
since the array of booleans uses 12 times less memory than `MPoly`.
It also allows faces to be stored more efficiently in the future, since
the flag is now unused. It's also possible to use generic functions to
process the values. For example, finding whether there is a sharp face
is just `sharp_faces.contains(true)`.
The `shade_smooth` attribute is no longer accessible with geometry nodes.
Since there were dedicated accessor nodes for that data, that shouldn't
be a problem. That's difficult to version automatically since the named
attribute nodes could be used in arbitrary combinations.
**Implementation notes:**
- The attribute and array variables in the code use the `sharp_faces`
term, to be consistent with the user-facing "sharp faces" wording,
and to avoid requiring many renames when #101689 is implemented.
- Cycles now accesses smooth face status with the generic attribute,
to avoid overhead.
- Changing the zero-value from "smooth" to "flat" takes some care to
make sure defaults are the same.
- Versioning for the edge mode extrude node is particularly complex.
New nodes are added by versioning to propagate the attribute in its
old inverted state.
- A lot of access is still done through the `CustomData` API rather
than the attribute API because of a few functions. That can be
cleaned up easily in the future.
- In the future we would benefit from a way to store attributes as a
single value for when all faces are sharp.
Pull Request: https://projects.blender.org/blender/blender/pulls/104422
If the texture image path in the MTL is a "quoted" absolute path, the importer will fail to find the
file. It was only attempting to un-quote the path for the relative case. Now we attempt to un-quote
in all cases.
Pull Request #105478
If the texture image path in the MTL is a "quoted" absolute path, the importer will fail to find the
file. It was only attempting to un-quote the path for the relative case. Now we attempt to un-quote
in all cases.
Pull Request #105478
With the goal of clearly differentiating between arrays and single
elements, improving consistency across Blender, and using wording
that's easier to read and say, change variable names for Mesh edges
and polygons/faces.
Common renames are the following, with some extra prefixes, etc.
- `mpoly` -> `polys`
- `mpoly`/`mp`/`p` -> `poly`
- `medge` -> `edges`
- `med`/`ed`/`e` -> `edge`
`MLoop` variables aren't affected because they will be replaced
when they're split up into to arrays in #104424.
Originally caused by 6514bb05ea
More cases where the active/default color attributes were not set
correctly:
[1] Using the old Python vertex_colors API (vertex_colors.new)
[2] OBJ importer
[3] Collada importer
[4] Data Transfer layout (both standalone operator and "Generate Data
Layers" from the modifier)
Similar to 101d04f41f.
Brought over from https://archive.blender.org/developer/D16977, see
discussion there why some of the code for data transfer is not for the
genereal attribute API.
Pull Request #105020
The logic for looping over imported OBJ faces and checking whether any
of them are "invalid" (duplicate vertices) was wrongly skipping
validation of the next face right after some invalid face. It
was the previously invalid face, moving the last into its place,
but then the loop was incrementing the face index and that just-moved
face was not properly validated.
Fixes#104593 - importing attached obj file (which contains some faces
that have duplicate indices). Added test coverage with a much smaller
obj file.
The OBJ spec (page B1-17) allows "l" entries to specify
polylines with more than 2 vertices, optionally with texture
coordinates.
Previously, only the first 2 vertices of each polyline
were read and added as loose edges, failing when texture
coordinates were present.
This adds support for proper polylines, reading but ignoring
texture coordinates.
Pull Request #104503
The new C++ OBJ importer was missing "split by objects" / "split by
groups" import settings of the older Python importer.
Implements T103839.
Added test coverage for all 4 possible combinations of these two
options.
Currently the `MLoopUV` struct stores UV coordinates and flags related
to editing UV maps in the UV editor. This patch changes the coordinates
to use the generic 2D vector type, and moves the flags into three
separate boolean attributes. This follows the design in T95965, with
the ultimate intention of simplifying code and improving performance.
Importantly, the change allows exporters and renderers to use UVs
"touched" by geometry nodes, which only creates generic attributes.
It also allows geometry nodes to create "proper" UV maps from scratch,
though only with the Store Named Attribute node for now.
The new design considers any 2D vector attribute on the corner domain
to be a UV map. In the future, they might be distinguished from regular
2D vectors with attribute metadata, which may be helpful because they
are often interpolated differently.
Most of the code changes deal with passing around UV BMesh custom data
offsets and tracking the boolean "sublayers". The boolean layers are
use the following prefixes for attribute names: vert selection: `.vs.`,
edge selection: `.es.`, pinning: `.pn.`. Currently these are short to
avoid using up the maximum length of attribute names. To accommodate
for these 4 extra characters, the name length limit is enlarged to 68
bytes, while the maximum user settable name length is still 64 bytes.
Unfortunately Python/RNA API access to the UV flag data becomes slower.
Accessing the boolean layers directly is be better for performance in
general.
Like the other mesh SoA refactors, backward and forward compatibility
aren't affected, and won't be changed until 4.0. We pay for that by
making mesh reading and writing more expensive with conversions.
Resolves T85962
Differential Revision: https://developer.blender.org/D14365
**Changes**
As described in T93602, this patch removes all use of the `MVert`
struct, replacing it with a generic named attribute with the name
`"position"`, consistent with other geometry types.
Variable names have been changed from `verts` to `positions`, to align
with the attribute name and the more generic design (positions are not
vertices, they are just an attribute stored on the point domain).
This change is made possible by previous commits that moved all other
data out of `MVert` to runtime data or other generic attributes. What
remains is mostly a simple type change. Though, the type still shows up
859 times, so the patch is quite large.
One compromise is that now `CD_MASK_BAREMESH` now contains
`CD_PROP_FLOAT3`. With the general move towards generic attributes
over custom data types, we are removing use of these type masks anyway.
**Benefits**
The most obvious benefit is reduced memory usage and the benefits
that brings in memory-bound situations. `float3` is only 3 bytes, in
comparison to `MVert` which was 4. When there are millions of vertices
this starts to matter more.
The other benefits come from using a more generic type. Instead of
writing algorithms specifically for `MVert`, code can just use arrays
of vectors. This will allow eliminating many temporary arrays or
wrappers used to extract positions.
Many possible improvements aren't implemented in this patch, though
I did switch simplify or remove the process of creating temporary
position arrays in a few places.
The design clarity that "positions are just another attribute" brings
allows removing explicit copying of vertices in some procedural
operations-- they are just processed like most other attributes.
**Performance**
This touches so many areas that it's hard to benchmark exhaustively,
but I observed some areas as examples.
* The mesh line node with 4 million count was 1.5x (8ms to 12ms) faster.
* The Spring splash screen went from ~4.3 to ~4.5 fps.
* The subdivision surface modifier/node was slightly faster
RNA access through Python may be slightly slower, since now we need
a name lookup instead of just a custom data type lookup for each index.
**Future Improvements**
* Remove uses of "vert_coords" functions:
* `BKE_mesh_vert_coords_alloc`
* `BKE_mesh_vert_coords_get`
* `BKE_mesh_vert_coords_apply{_with_mat4}`
* Remove more hidden copying of positions
* General simplification now possible in many areas
* Convert more code to C++ to use `float3` instead of `float[3]`
* Currently `reinterpret_cast` is used for those C-API functions
Differential Revision: https://developer.blender.org/D15982
Some files out there (e.g. in T103441) contain face definitions with
four indices, which the importer code was not expecting. The OBJ
standard spells out up to three indices per face corner; the file in
there must be using some sort of non-standard OBJ syntax extension.
Now the code simply ignores any trailing per-face-corner data.
Similar to T98782 that was about normal indices: some files out there
are technically "wrong" since they refer to UV indices, without ever
defining any UVs. If the file does not have any UVs, simply ignore UV
indices in the OBJ face definitions. Fixes T103212.
Attributes are unifying around a name-based API, and we would like to
be able to move away from CustomData in the future. This patch moves
the identification of active and fallback (render) color attributes
to strings on the mesh from flags on CustomDataLayer. This also
removes some ugliness used to retrieve these attributes and maintain
the active status.
The design is described more here: T98366
The patch keeps forward compatibility working until 4.0 with
the same method as the mesh struct of array refactors (T95965).
The strings are allowed to not correspond to an attribute, to allow
setting the active/default attribute independently of actually filling
its data. When applying a modifier, if the strings don't match an
attribute, they will be removed.
The realize instances / join node and join operator take the names from
the first / active input mesh. While other heuristics may be helpful
(and could be a future improvement), just using the first is simple
and predictable.
Differential Revision: https://developer.blender.org/D15169
As part of T95966, this patch moves loose edge information out of the
flag on each edge and into a new lazily calculated cache in mesh
runtime data. The number of loose edges is also cached, so further
processing can be skipped completely when there are no loose edges.
Previously the `ME_LOOSEEDGE` flag was updated on a "best effort"
basis. In order to be sure that it was correct, you had to be sure
to call `BKE_mesh_calc_edges_loose` first. Now the loose edge tag
is always correct. It also doesn't have to be calculated eagerly
in various places like the screw modifier where the complexity
wasn't worth the theoretical performance benefit.
The patch also adds a function to eagerly set the number of loose
edges to zero to avoid building the cache. This is used by various
primitive nodes, with the goal of improving drawing performance.
This results in a few ms shaved off extracting draw data for some
large meshes in my tests.
In the Python API, `MeshEdge.is_loose` is no longer editable.
No built-in addons set the value anyway. The upside is that
addons can be sure the data is correct based on the mesh.
**Tests**
There is one test failure in the Python OBJ exporter: `export_obj_cube`
that happens because of existing incorrect versioning. Opening the
file in master, all the edges were set to "loose", which is fixed
by this patch.
Differential Revision: https://developer.blender.org/D16504
Using varargs had the disadvantages, replace with a macro which has
some advantages.
- Arguments are type checked.
- Less verbose.
- Unintended NULL arguments would silently terminate joining paths.
- Passing in a NULL argument warns with GCC.
Part of T101073: after the view layer sync was made lazy (D15885), the
BKE_layer_collection_resync_forbid and BKE_layer_collection_resync_allow
in Alembic/USD/OBJ importers is no longer needed, as long as they do
view layer dependent operations (selecting new objects) in a separate
loop after creating all the objects.
Verified that this does not regress import times for 26k objects OBJ
scene (Blender 3.0 splash) and 250k objects USD scene (Moana).
Requested in D16095 proposal - also USD & Alembic have import scale
option; OBJ has an export scale object but the import scale
was not there for some reason.
The importer logic was wrongly resetting "current material name"
upon encountering a new object ("o" command). However as per OBJ
specification, this is incorrect:
> Specifies the material name for the element following it. Once a
> material is assigned, it cannot be turned off; it can only be
> changed.
Fixes T101685. Test coverage for this was added in svn tests repo.
The UV data filling logic was incorrectly just skipping over loop
entries that don't have a UV coordinate, instead of assigning
the default zero UV for them. This was a problem only for meshes
where some faces did have UVs, but some other faces did not (T101487).
When a change happens which invalidates view layers the syncing will be postponed until the first usage.
This will improve importing or adding many objects in a single operation/script.
`BKE_view_layer_need_resync_tag` is used to tag the view layer to be out of sync. Before accessing
`BKE_view_layer_active_base_get`, `BKE_view_layer_active_object_get`, `BKE_view_layer_active_collection`
or `BKE_view_layer_object_bases` the caller should call `BKE_view_layer_synced_ensure`.
Having two functions ensures that partial syncing could be added as smaller patches in the future. Tagging a
view layer out of sync could be replaced with a partial sync. Eventually the number of full resyncs could be
reduced. After all tagging has been replaced with partial syncs the ensure_sync could be phased out.
This patch has been added to discuss the details and consequences of the current approach. For clarity
the call to BKE_view_layer_ensure_sync is placed close to the getters.
In the future this could be placed in more strategical places to reduce the number of calls or improve
performance. Finding those strategical places isn't that clear. When multiple operations are grouped
in a single script you might want to always check for resync.
Some areas found that can be improved. This list isn't complete.
These areas aren't addressed by this patch as these changes would be hard to detect to the reviewer.
The idea is to add changes to these areas as a separate patch. It might be that the initial commit would reduce
performance compared to master, but will be fixed by the additional patches.
**Object duplication**
During object duplication the syncing is temporarily disabled. With this patch this isn't useful as when disabled
the view_layer is accessed to locate bases. This can be improved by first locating the source bases, then duplicate
and sync and locate the new bases. Will be solved in a separate patch for clarity reasons ({D15886}).
**Object add**
`BKE_object_add` not only adds a new object, but also selects and activates the new base. This requires the
view_layer to be resynced. Some callers reverse the selection and activation (See `get_new_constraint_target`).
We should make the selection and activation optional. This would make it possible to add multiple objects
without having to resync per object.
**Postpone Activate Base**
Setting the basact is done in many locations. They follow a rule as after an action find the base and set
the basact. Finding the base could require a resync. The idea is to store in the view_layer the object which
base will be set in the basact during the next sync, reducing the times resyncing needs to happen.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15885
Related to {D15885} that requires scene parameter
to be added in many places. To speed up the review process
the adding of the scene parameter was added in a separate
patch.
Reviewed By: mont29
Maniphest Tasks: T73411
Differential Revision: https://developer.blender.org/D15930
Implement import & export support for "PBR extensions" in .mtl files
(T101029, also fixes T86736).
Newly supported parameters:
- Roughness (Pr, map_Pr)
- Metallic (Pm, map_Pm)
- Sheen (Ps, map_Ps)
- Clearcoat thickness (Pc) and roughness (Pcr)
- Anisotropy (aniso) and rotation (anisor)
- Transmittance (Tf / Kt)
Exporter has an option to enable these additional PBR parameters
export; defaults to off since not all software understands that.
Exporter UI tweaked and all material-related options were put into
their own separate box.
Added/extended test files in Subversion repository for test coverage.
The members were named after .mtl file syntax ("d") instead of their
meaning ("alpha"). In preparation for extending OBJ code for more
PBR parameters support, rename them for clarity. No functionality
changes, just a pure rename.
- Remove "take ownership" argument which was confusing and always true
- The argument made ownership very confusing
- Better to avoid boolean arguments that switch a function's purpose
- Remove "mask" argument which was basically wrong and not used properly
- "EVERYTHING" was used because developers are wary of removing data
- Instead use `CD_MASK_MESH` for its purpose of original mesh data
- Remove use of shallow copied temporary mesh, which is unnecessary now
- Split shape key processing into separate functions and use C++ types
- Copy fields explicitly rather than using memcpy for the whole struct
- Use higher level functions and avoid redundant code
- The whole idea is pretty simple and can be built from standard logic
- Adjust `CustomData` logic to be consistent with "assign" expectations
- Clear the layer data from the source, and moves the anonymous ID
Differential Revision: https://developer.blender.org/D15857
Replace `mesh_attributes`, `mesh_attributes_for_write` and the point
cloud versions with methods on the `Mesh` and `PointCloud` types.
This makes them friendlier to use and improves readability.
Differential Revision: https://developer.blender.org/D15907
Use `verts` instead of `vertices` and `polys` instead of `polygons`
in the API added in 05952aa94d. This aligns better with
existing naming where the shorter names are much more common.
For copy-on-write, we want to share attribute arrays between meshes
where possible. Mutable pointers like `Mesh.mvert` make that difficult
by making ownership vague. They also make code more complex by adding
redundancy.
The simplest solution is just removing them and retrieving layers from
`CustomData` as needed. Similar changes have already been applied to
curves and point clouds (e9f82d3dc7, 410a6efb74). Removing use of
the pointers generally makes code more obvious and more reusable.
Mesh data is now accessed with a C++ API (`Mesh::edges()` or
`Mesh::edges_for_write()`), and a C API (`BKE_mesh_edges(mesh)`).
The CoW changes this commit makes possible are described in T95845
and T95842, and started in D14139 and D14140. The change also simplifies
the ongoing mesh struct-of-array refactors from T95965.
**RNA/Python Access Performance**
Theoretically, accessing mesh elements with the RNA API may become
slower, since the layer needs to be found on every random access.
However, overhead is already high enough that this doesn't make a
noticible differenc, and performance is actually improved in some
cases. Random access can be up to 10% faster, but other situations
might be a bit slower. Generally using `foreach_get/set` are the best
way to improve performance. See the differential revision for more
discussion about Python performance.
Cycles has been updated to use raw pointers and the internal Blender
mesh types, mostly because there is no sense in having this overhead
when it's already compiled with Blender. In my tests this roughly
halves the Cycles mesh creation time (0.19s to 0.10s for a 1 million
face grid).
Differential Revision: https://developer.blender.org/D15488
Add a dedicated `owner_id` pointer to ID types that can be embedded
(Collections and NodeTrees), and modify slightly come code to make
handling those more safe and consistent.
This implements first part of T69169.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D15838
Previously for most materials (especially the ones without any
textures), the nodes were "off screen" way to the right, requiring
a view framing to even see them.
Also, as soon as multiple images were used, many nodes overlapped
one another and the connections were all a mess.
Simplify all that, and now each node type (coordinate, mapping, image,
normal map, bsdf etc.) is in it's own column, with BSDF at zero
coordinate. Each used image (along with any possible coordinate,
mapping, normal map) is it's own row. The resulting connections
are much cleaner.
As pointed out in D15827 comment, the unique_ptr usage in
ShaderNodetreeWrap related code does not sound very useful. Looking at
it, whole ShaderNodetreeWrap does not make much sense - it's only
ever created, and then immediately just one thing is fetched from it.
This very much sounds like "a function", so make it just that -
header file contains just a `create_mtl_node_tree` function, and the
whole implementation is hidden from the users. Which I've also
simplified into just a handful of freestanding functions.
No functionality or performance changes, but the code does get ~80
lines shorter.
Fixes issues in importers written in C++ (T100737):
- Materials had one reference count too much. Affected Collada,
Alembic, USD, OBJ importers, looks like "since forever".
- Active material index was not properly set on imported meshes.
Regression since 3.3 (D15145). Affected Alembic, USD, OBJ. Note:
now it sets the first material as the active one, whereas
previously the last one was set as active. First one sounds more
"intuitive" to me.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15831
