Report T98781 and part of T97642: the MTLMaterial info only captures
image nodes and the default socket values. When the image information
is present, do not emit the socket defaults - the .MTL spec states
they are multiplied together, but the default value is not used
in blender when the socket is connected.
Also contains svn tests repository update to extend the test coverage,
and update test expectation outputs.
While T77801 itself is working as expected in the new C++ obj
importer, the repro file there uses absolute paths to material images,
yet the images themselves are right there in the current folder.
The old python based importer did find them, since it was doing a
really complex image search. My understanding is that while C++
importer was developed, it was decided to not do that -- however
just the "basename file in the mtl directory" sounds simple enough
and gets the repro case file work correctly.
As part of the previous fix (D15410), the importer got code to track
min & max vertex indices used as part of the mesh faces. However, if
faces refer to a "sparse" (i.e. non-contiguous) subset of all vertices,
then the imported mesh would contain all the vertices between min & max
range.
Replace that with proper tracking of actually used vertex indices
for each imported mesh. Fixes T100302.
This does affect import performance a tiny bit, e.g. importing Blender
3.0 splash scene goes 21.7s -> 22.1s, and importing rungholt.obj
goes 2.37s -> 2.48s.
Importer related tests have a bunch of vertex changes in them, since
now vertices are added in the order that the faces are referring
to them. Which incidentally matches the order that the Python based
importer was creating them too.
NOTE: This is committed to the 3.3 branch as part of D15606, which we
decided should go to this release still (by Bastien, Dalai and me). That
is because these are important usability fixes/improvements to have for
the LTS release.
Adds `rna_path.cc` and `RNA_path.h`.
`rna_access.c` is a quite big file, which makes it rather hard and
inconvenient to navigate. RNA path functions form a nicely coherent unit
that can stand well on it's own, so it makes sense to split them off to
mitigate the problem. Moreover, I was looking into refactoring the quite
convoluted/overloaded `rna_path_parse()`, and found that some C++
features may help greatly with that. So having that code compile in C++
would be helpful to attempt that.
Differential Revision: https://developer.blender.org/D15540
Reviewed by: Brecht Van Lommel, Campbell Barton, Bastien Montagne
The Python based exporter was replacing spaces with underscores
in object/group names, mostly to handle cases where names could begin
or end with spaces. The new exporter was not doing that. Note: spaces
in material names were already handled by the new exporter.
Fixes T97769. Updated test coverage expectations; one of the test
files has an object with a space in the name.
Since 1a81d268a1, materials on object data can change during
evaluation. But a different function is necessary to retrieve materials
taking that into account.
Solves part of T96721.
Differential Revision: https://developer.blender.org/D15595
Python based OBJ importer, as well as glTF2 importer, are creating
"placeholder" images for texture images that can't be found. These
are empty textures (displayed as magenta), but with their file paths
set so that File > External Data > Report Missing Files can report
them as missing.
Make the new C++ OBJ importer do the same as well. Fixes T99502.
The Python based importer had logic to immediately turn image paths
into relative-to-blender-file paths, if user preference for relative
paths is used (which is on by default). The new importer code did not
have that. Fixes T100076.
The new OBJ/MTL importer was creating a new image for any referenced
texture, even if another material (or another property of the same
material) already referenced the same texture. Make it use
BKE_image_load_exists function just like Collada or USD importers do.
Fixes T100075. Extended test coverage to count imported images;
without the fix import_cubes_with_textures_rel would have incorrectly
created 5 images instead of 4.
The Python based importer had a special case handling of "no faces in
the whole file at all", where it ended up treating the whole file
as essentially a point-cloud-like object (just loose vertices, no
faces or edges). The new importer code was missing this special case.
Fixes T100017. Added gtest coverage that was failing without the fix.
The Alembic importer uses a linear search over the mesh edges to find
the right edge when setting edge creases. Although the complexity is
`O(m * n)`, with `m` being the number of creased edges, and `n` being
the number of edges, this can lead to a quadratic complexity as `m`
approches `n`.
This patch uses `EdgeHash` to store and retrieve the edges, which
should bring complexity closer to `O(n)`, provided that lookup is
`O(1)`.
See differential for some timings. In most files, this is expected
to give at least a 2-3x speedup for this operation, but can lead
orders of magnitude speed increase for dense meshes with a significant
number of edge creases.
Differential Revision: https://developer.blender.org/D15521
Previously, when creating "very large" (tens-hundreds of thousands)
amounts of objects, the Blender code that was ensuring name
uniqueness was the bottleneck. That got recently addressed (D14162),
however now sorting of IDs by their names is the remaining bottleneck.
Name sorting code in Blender is optimized for the pattern where names
are inserted in already sorted order (i.e. objects expect to get added
near the end of the list). By doing this pre-sorting of objects
intended to get created by an importer (USD and OBJ, in this patch),
this sorting bottleneck can be largely removed, especially with very
high object counts.
Windows, Ryzen 5950X, import times:
- OBJ, splash screen scene (26k objects): 22.0s -> 20.7s
- USD, Disney Moana scene (250k objects): 585s -> 82.2s (10 minutes -> 1.5 minutes)
Reviewed By: Michael Kowalski, Howard Trickey
Differential Revision: https://developer.blender.org/D15506
Rename and refactor several F-curve key manipulation functions, and move
them from `editors` to `blenkernel`.
The functions formerly known as `delete_fcurve_key`,
`delete_fcurve_keys`, and `clear_fcurve_keys` have been moved from
`ED_keyframes_edit.h` to `BKE_fcurve.h` and have been renamed according
to hierarchical naming rules.
Below is a table of the naming changes.
| From | To |
| -- | -- |
| `delete_fcurve_key(fcu, index, do_recalc)` | `BKE_fcurve_delete_key(fcu, index)` |
| `delete_fcurve_keys(fcu)` | `BKE_fcurve_delete_keys_selected(fcu)` |
| `clear_fcurve_keys(fcu)` | `BKE_fcurve_delete_keys_all(fcu)` |
| `calchandles_fcurve()` | `BKE_fcurve_handles_recalc()` |
| `calchandles_fcurve_ex()`| `BKE_fcurve_handles_recalc_ex()` |
The function formerly known as `delete_fcurve_key` no longer takes a
`do_fast` parameter, which determined whether or not to call
`calchandles_fcurve`. Now, the responsibility is on the caller to run
the new `BKE_fcurve_handles_recalc` function if they have want to
recalculate the handles.
In addition, there is now a new static private function called
`fcurve_bezt_free` which sets the key count to zero and frees the key
array. This function is now used in couple of instances of functionally
equivalent code. Note that `BKE_fcurve_delete_keys_all` is just a
wrapper around `fcurve_bezt_free`.
This change was initially spurred by the fact that `delete_fcurve_keys`
was improperly named; this was a good opportunity to fix the location
and naming of a few of these functions.
Reviewed By: sybren
Differential Revision: https://developer.blender.org/D15282
The importer code was written under incorrect assumption that vertex
data (v, vn, vt commands etc.) are grouped by object, i.e. follow the
o command, and that each object has its own vertex data commands. This
is not the case -- all the vertex data in the whole OBJ file is
"global", with no relation to any objects/groups; it's just that the
faces belong to the object, and then they pull in any vertices they
like.
This patch fixes this incorrect assumption in the importer:
- Vertex data is now properly global; no need to track some sort of
"offsets" per object like it was doing before.
- For each object, face definitions track the minimum & maximum vertex
indices referenced by the object, and then all that vertex range is
created in the final Blender object. Note: it might be (unusual, but
possible) that an object does not reference a sequential range of
vertices, e.g. just a single face with vertex indices 1, 10, 100 --
the resulting Blender mesh will have all the 100 vertices (some
"loose" without belonging to a face). It should be possible to track
the used vertices exactly (e.g. with a vector set), but I haven't
done that for performance reasons.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15410
Address the issue by re-working line continuation handling: stop
trying to parse sequences like "backslash, newline" (which is the
bug: it should also handle "backslash, possible whitespace, newline")
during parsing. Instead, fixup line continuations after reading chunks
of input file data - turn backslash and the following newline into
spaces. The rest of parsing code does not have to be aware of them
at all then.
Makes the file attached to T99536 load correctly now. Also will extend
one of the test files in subversion tests repo to contain backslashes
followed by newlines.
Currently, there are two attribute API. The first, defined in `BKE_attribute.h` is
accessible from RNA and C code. The second is implemented with `GeometryComponent`
and is only accessible in C++ code. The second is widely used, but only being
accessible through the `GeometrySet` API makes it awkward to use, and even impossible
for types that don't correspond directly to a geometry component like `CurvesGeometry`.
This patch adds a new attribute API, designed to replace the `GeometryComponent`
attribute API now, and to eventually replace or be the basis of the other one.
The basic idea is that there is an `AttributeAccessor` class that allows code to
interact with a set of attributes owned by some geometry. The accessor itself has
no ownership. `AttributeAccessor` is a simple type that can be passed around by
value. That makes it easy to return it from functions and to store it in containers.
For const-correctness, there is also a `MutableAttributeAccessor` that allows
changing individual and can add or remove attributes.
Currently, `AttributeAccessor` is composed of two pointers. The first is a pointer
to the owner of the attribute data. The second is a pointer to a struct with
function pointers, that is similar to a virtual function table. The functions
know how to access attributes on the owner.
The actual attribute access for geometries is still implemented with the `AttributeProvider`
pattern, which makes it easy to support different sources of attributes on a
geometry and simplifies dealing with built-in attributes.
There are different ways to get an attribute accessor for a geometry:
* `GeometryComponent.attributes()`
* `CurvesGeometry.attributes()`
* `bke::mesh_attributes(const Mesh &)`
* `bke::pointcloud_attributes(const PointCloud &)`
All of these also have a `_for_write` variant that returns a `MutabelAttributeAccessor`.
Differential Revision: https://developer.blender.org/D15280
It's helpful to make the separation of legacy data formats explicit,
because it declutters actively changed code and makes it clear which
areas do not follow Blender's current design. In this case I separated
the `MFace`/"tessface" conversion code into a separate blenkernel
.cc file and header. This also makes refactoring to remove these
functions simpler because they're easier to find.
In the future, conversions to the `MLoopUV` type and `MVert`
can be implemented here for the same reasons (see T95965).
Differential Revision: https://developer.blender.org/D15396
As pointed out in a comment on T89421, if a MTL file contained
something like: `map_Ka -o 1 2.png` then it was parsed as having
offset `1 2` and the texture filename just a `.png`. Make it so that
mtl option numbers are parsed in a way where the number is only
accepted only if it's followed by whitespace.
Differential Revision: https://developer.blender.org/D15385
Fixes T97743: the import code was setting EEVEE blending mode whenever
a transparency texture was present (map_d), or when the materials
illum was saying "yo, transparency!". But if only the material's d
was below 1.0, it was not setting the blend mode, which is different
to user expectations.
Differential Revision: https://developer.blender.org/D15383
The importer parts that were doing assignment of materials to the
imported objects/meshes were essentially having a quadratic complexity
in terms of scene object count. For each material assigned to each
object, they were scanning the whole scene, checking which other
Objects use the same Mesh data, in order to resize their material
arrays to match the size.
Performance details (Windows, Ryzen 5950X):
- Import OBJ Blender 3.0 splash scene (24k objects): 43.0s -> 32.9s
- Import USD Disney Moana scene (260k objects): saves two hours
(~7400s). Note that later on this crashes when trying to render the
imported result; crashes in the same way/place both in master and
this patch.
Implementation details:
The importers were doing "scan the world" basically twice for each
object, for each material: once when creating a new material slot
(assigns an empty material), and then again when assigning the
material.
However, all these importers (USD, Alembic, OBJ) always create one
Object for one Mesh. So that whole quadratic complexity resulting
from "scan the world for possible other users of this obdata" is
completely not needed; it just never finds anything. So add a new
dedicated function BKE_object_material_assign_single_obdata that skips
the expensive part, but should only be used when the caller knows that
the obdata has exactly one user (the passed object).
Reviewed By: Bastien Montagne, Michael Kowalski
Differential Revision: https://developer.blender.org/D15145
The new OBJ/MTL importer was already handling case T89421
correctly, but there was no test coverage to prove it. Extend
the tests to parse various forms of "-o" and "-s" (one, two, three
numbers).
Many existing importers/exporters do log the time it takes to system
console (some others log more information too). In particular, OBJ
(C++ & python), STL (C++ & python), PLY, glTF2 all log the time it
takes. However, neither USD nor Alembic do. And also it's harder to
know the time it takes there from a profiler, since all the work
normally is done on a background job and is split between several
threads (so you can't just find some top-level function and see how
much time it took).
This change:
- Adds import/export time logging to USD & Alembic importer/exporter,
- In the time utility class (also used by OBJ & STL), improve the
output formatting: 1) print only one decimal digit, 2) for long
times, print seconds and also produce a hours:minutes:seconds form.
Reviewed By: Michael Kowalski, Kévin Dietrich
Differential Revision: https://developer.blender.org/D15170
Many thanks to the original Author of this patch: Christian Aguilera
The COLLADA importer was silently ignoring the alpha component in the
vertex data.
The `stride` variable holds the component count (3 for RGB; 4 for RGBA),
and can be used for honouring the alpha channel in the vertex data.
Test plan:
- Open Blender.
- Clear the scene.
- Add a plane.
- Enter **Vertex Paint** mode.
- Switch to the **Erase Alpha** blending mode.
- Select a tone of gray.
- Turn strength down to less than 1
- Paint [some of] the vertices of the plane.
- Export project as a COLLADA file (`.dae`).
- Clear the scene.
- Re-import the COLLADA file again.
- Export the project again (with different name).
**Without** this patch, the second exported project will have lost the
alpha component in their vertex data:
```lang=xml, counterexample
<float_array id="Plane-mesh-colors-Col-array" count="24">1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1</float_array>
```
**With** the patch, the first and the second exported projects retain
the alpha values painted previously:
```lang=xml
<float_array id="Plane-mesh-colors-Col-array" count="24">1 1 1 1 1 1 1 0.5490196 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0.5490196</float_array>
```
Reviewed By: cristian64, SonnyCampbell_Unity
Authored by: Christian Aguilera
Differential Revision: https://developer.blender.org/D14246
Removes the following macros for scene/render frame values:
- `CFRA`
- `SUBFRA`
- `SFRA`
- `EFRA`
These macros don't add much, other than saving a few characters when typing.
It's not immediately clear what they refer to, they just hide what they
actually access. Just be explicit and clear about that.
Plus these macros gave read and write access to the variables, so eyesores like
this would be done (eyesore because it looks like assigning to a constant):
```
CFRA = some_frame_nbr;
```
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D15311
Fix unused function result warnings for usages of `fread` in the C++
.stl importer, by actually using the returned error code.
Reviewed By: sybren
Differential Revision: https://developer.blender.org/D15189
Fixes C++ .stl importer info output having no space between the
number and the word after it.
Reviewed By: Aras Pranckevicius
Differential Revision: https://developer.blender.org/D15240
The old Python OBJ importer had a (somewhat confusingly named) "Keep
Vertex Order -> Poly Groups" option, that imported OBJ groups as
"vertex groups" on the resulting mesh. All vertices of any face were
assigned the vertex group, with a 1.0 weight.
The new C++ importer did not have this option. It was trying to do
something with vertex groups, but failing to actually achieve
anything :) -- the vertex groups were created on the wrong object
(later on overwritten by "nomain mesh to main mesh" operation);
vertex weights were set to 1.0/vertex_count, and each vertex was only
set to be in one group, even when it belongs to multiple faces from
different groups. End result was that to the user, vertex groups were
not visible/present at all (see T98874).
This patch adds the import option (named "Vertex Groups"), which is
off by default, and fixes the import code logic to actually do the
right thing. Tested on file from T98874; vertex groups are imported
just like with the Python importer.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15200
The new OBJ importer is producing "sharp" edges on some meshes that
should be completely smooth. Only observed on UV-Sphere type meshes
so far (see T97820).
I'm not 100% sure what is the root cause, but my theory was that
maybe due to limited number of float digits that are printed for
vertex normals in the file, the normals that are read in are not
always exactly 1.0 length. And then the Blender's "set custom loop
normals" function (which expects normalized inputs) wrongly marks
some edges as sharp.
Adding explicit normalization for the normals that are read from the
file fixes the wrongly-sharp edges in test cases from T97820. I
have not observed measurable performance impact in importing large
models (e.g. 6-level subdivided Monkey) that contain vertex normals.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15202
Previous code was rebuilding "name to material" map for each object
being imported. Which means O(N*M) complexity (N=object count,
M=material count). There was already a TODO comment suggesting that
a single map that's maintained for the whole import would be enough.
This commit does exactly that.
While importing Moana USD scene (260k objects, 18k materials) this
saves about 6 minutes of import time.
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15222
Previous code was doing N collection syncs when importing N objects
(essentially quadratic complexity in terms of object count). New
code avoids all the intermediate syncs by using
BKE_layer_collection_resync_forbid and
BKE_layer_collection_resync_allow, and then does one
BKE_main_collection_sync + BKE_main_collection_sync_remap for the
whole operation. The things done on the importer objects that are
dependent on the sync happening (marking them selected) are done in a
separate loop after the sync.
Timings: importing Moana USD scene (480k objects) on Windows, VS2022
Release build, AMD Ryzen 5950X: 12344sec -> 10979sec (saves 22 minutes).
Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D15215
Some I/O code paths (Collada, OBJ) were using mat3_from_axis_conversion
followed by transpose_m3, instead of swapping the axis arguments
which achieves exactly the same result.
Reviewed By: Aras Pranckevicius
Differential Revision: https://developer.blender.org/D15158
OBJ vertex color related tests were not producing identical results
across various platforms, primarily due to sRGB<->Linear color space
conversions.
While D15193 has just made the color space conversion accuracy match
much closer between platforms, it's still not 100% the same.
This change reduces the amount of decimal places used for exporting
vertex colors, to 4 digits (down from 6). Vertex normals were
already always printed with 4 digits, and colors are conceptually
similar (usually 0..1 range etc.).
This makes the vertex color tests pass again, so re-enable them
after adjusting to 4 decimals expectations.
Ensure the "null" node graph, which is the root node of the export
graph, always exists.
The crash occured when "Use Settings For" was set to Render, "Visible
Objects Only" was ticked, and a single parent object is in the scene but
disabled for render.
Because the only object attached to the root of the project was disabled
for export, there was no "null" root node added to the export graph.
This change will always add an empty "null" node with no children to the
graph at the start. Other objects will get added to its children as
required.
Reviewed By: sybren
Maniphest Tasks: T85729
Differential Revision: https://developer.blender.org/D15182
Designate private variable names as described by the style guide,
and also add `num` at the end of variable names rather than at
the beginning, as discussed in T85728.
Some OBJ files out there (see T98782) have face definitions that
contain vertex normal indices, but the files themselves don't
contain any vertex normals. The code was doing a "hey, that's an
invalid index" and skipping these faces. But the old python importer
was silently ignoring these normal indices, so do the same here.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15177
Adds support for vertex colors to OBJ I/O.
Importer:
- Supports both "xyzrgb" and "MRGB" vertex color formats.
- Whenever vertex color is present in the file for a model, it is
imported and a Color attribute is created (per-vertex, full float
color data type). Color coming from the file is assumed to be sRGB,
and is converted to linear upon import.
Exporter:
- Option to export the vertex colors. Defaults to "off", since not
all 3rd party software supports vertex colors.
- When the option is "on", if a mesh has a color attribute layer,
the active one is exported in "xyzrgb" form. If the mesh has
per-face-corner colors, they are averaged on the vertices.
Colors are converted from linear to sRGB upon export.
Reviewed By: Howard Trickey
Differential Revision: https://developer.blender.org/D15159
The documentation for `HierarchyIterator::weak_export` mentions a feature
that was removed at some point. Another example is used to illustrate its
functionality.
No functional changes.
This is a partial fix for T90535.
Added Material Name Collision USD import menu option, to specify
the behavior when USD materials in different namespaces have the
same name.
The Material Name Collision menu options are
- Make Unique: Import each USD material as a unique Blender material.
- Reference Existing: If a material with the same name already
exists, reference that instead of importing.
Previously, the default behavior was to always keep the existing
material. This was causing an issue in the ALab scene, where
dozens of different USD materials all have the same name,
usdpreviewsurface1, so that only one instance of these materials
would be imported.
Reviewed by: Sybren
Differential Revision: https://developer.blender.org/D14869
