213cd39b6d
Continued improvements to the new C++ based OBJ importer. Performance: about 2x faster. - Rungholt.obj (several meshes, 263MB file): Windows 12.7s -> 5.9s, Mac 7.7s -> 3.1s. - Blender 3.0 splash (24k meshes, 2.4GB file): Windows 97.3s -> 53.6s, Mac 137.3s -> 80.0s. - "Windows" is VS2022, AMD Ryzen 5950X (32 threads), "Mac" is Xcode/clang 13, M1Max (10 threads). - Slightly reduced memory usage during import as well. The performance gains are a combination of several things: - Replacing `std::stof` / `std::stoi` with C++17 `from_chars`. - Stop reading input file char-by-char using `std::getline`, and instead read in 64kb chunks, and parse from there (taking care of possibly handling lines split mid-way due to chunk boundaries). - Removing abstractions for splitting a line by some char, - Avoid tiny memory allocations: instead of storing a vector of polygon corners in each face, store all the corners in one big array, and per-face only store indices "where do corners start, and how many". Likewise, don't store full string names of material/group names for each face; only store indices into overall material/group names arrays. - Stop always doing mesh validation, which is slow. Do it just like the Alembic importer does: only do validation if found some invalid faces during import, or if requested by the user via an import setting checkbox (which defaults to off). - Stop doing "collection sync" for each object being added; instead do the collection sync right after creating all the objects. Cleanup / Robustness: This reworking of parser (see "removing abstractions" point above) means that all the functions that were in `parser_string_utils` file are gone, and replaced with different set of functions. However they are not OBJ specific, so as pointed out during review of the previous differential, they are now in `source/blender/io/common` library. Added gtest coverage for said functions as well; something that was only indirectly covered by obj tests previously. Rework of some bits of parsing made the parser actually better able to deal with invalid syntax. E.g. previously, if a face corner were a `/123` string, it would have incorrectly treated that as a vertex index (since it would get "hey that's one number" after splitting a string by a slash), instead of properly marking it as invalid syntax. Added gtest coverage for .mtl parsing; something that was not covered by any tests at all previously. Reviewed By: Howard Trickey Differential Revision: https://developer.blender.org/D14586
100 lines
2.3 KiB
C++
100 lines
2.3 KiB
C++
/* SPDX-License-Identifier: GPL-2.0-or-later */
|
|
|
|
#include "IO_string_utils.hh"
|
|
|
|
/* Note: we could use C++17 <charconv> from_chars to parse
|
|
* floats, but even if some compilers claim full support,
|
|
* their standard libraries are not quite there yet.
|
|
* LLVM/libc++ only has a float parser since LLVM 14,
|
|
* and gcc/libstdc++ since 11.1. So until at least these are
|
|
* the mininum spec, use an external library. */
|
|
#include "fast_float.h"
|
|
#include <charconv>
|
|
|
|
namespace blender::io {
|
|
|
|
StringRef read_next_line(StringRef &buffer)
|
|
{
|
|
const char *start = buffer.begin();
|
|
const char *end = buffer.end();
|
|
size_t len = 0;
|
|
char prev = 0;
|
|
const char *ptr = start;
|
|
while (ptr < end) {
|
|
char c = *ptr++;
|
|
if (c == '\n' && prev != '\\') {
|
|
break;
|
|
}
|
|
prev = c;
|
|
++len;
|
|
}
|
|
|
|
buffer = StringRef(ptr, end);
|
|
return StringRef(start, len);
|
|
}
|
|
|
|
static bool is_whitespace(char c)
|
|
{
|
|
return c <= ' ' || c == '\\';
|
|
}
|
|
|
|
StringRef drop_whitespace(StringRef str)
|
|
{
|
|
while (!str.is_empty() && is_whitespace(str[0])) {
|
|
str = str.drop_prefix(1);
|
|
}
|
|
return str;
|
|
}
|
|
|
|
StringRef drop_non_whitespace(StringRef str)
|
|
{
|
|
while (!str.is_empty() && !is_whitespace(str[0])) {
|
|
str = str.drop_prefix(1);
|
|
}
|
|
return str;
|
|
}
|
|
|
|
static StringRef drop_plus(StringRef str)
|
|
{
|
|
if (!str.is_empty() && str[0] == '+') {
|
|
str = str.drop_prefix(1);
|
|
}
|
|
return str;
|
|
}
|
|
|
|
StringRef parse_float(StringRef str, float fallback, float &dst, bool skip_space)
|
|
{
|
|
if (skip_space) {
|
|
str = drop_whitespace(str);
|
|
}
|
|
str = drop_plus(str);
|
|
fast_float::from_chars_result res = fast_float::from_chars(str.begin(), str.end(), dst);
|
|
if (res.ec == std::errc::invalid_argument || res.ec == std::errc::result_out_of_range) {
|
|
dst = fallback;
|
|
}
|
|
return StringRef(res.ptr, str.end());
|
|
}
|
|
|
|
StringRef parse_floats(StringRef str, float fallback, float *dst, int count)
|
|
{
|
|
for (int i = 0; i < count; ++i) {
|
|
str = parse_float(str, fallback, dst[i]);
|
|
}
|
|
return str;
|
|
}
|
|
|
|
StringRef parse_int(StringRef str, int fallback, int &dst, bool skip_space)
|
|
{
|
|
if (skip_space) {
|
|
str = drop_whitespace(str);
|
|
}
|
|
str = drop_plus(str);
|
|
std::from_chars_result res = std::from_chars(str.begin(), str.end(), dst);
|
|
if (res.ec == std::errc::invalid_argument || res.ec == std::errc::result_out_of_range) {
|
|
dst = fallback;
|
|
}
|
|
return StringRef(res.ptr, str.end());
|
|
}
|
|
|
|
} // namespace blender::io
|