Commit Graph

14 Commits

Author SHA1 Message Date
Jacques Lucke
c233955ef9 Geometry Nodes: support custom delimiter in Import CSV node
Previously, only `,` was a valid delimiter, but it's also common to have e.g.
tab or semicolon. Only single characters are supported as delimiters. A few
characters are not allowed because they have other meanings: `\n`, `\r`, `"`,
`\\`.

Note: The best way to get a tab character is the use the Special Characters node
currently.

Pull Request: https://projects.blender.org/blender/blender/pulls/135468
2025-03-04 23:58:02 +01:00
Campbell Barton
85af07f438 Cleanup: spelling in comments 2025-02-23 21:02:54 +11:00
Jacques Lucke
88157430c1 Geometry Nodes: eagerly update bounding box after CSV import
This avoids computing the bounding box later (if the positions are not changed).
Since the positions are are zero, the bounding box is just the origin point too.
2025-02-20 11:45:21 +01:00
Jacques Lucke
46ae0a86ca Geometry Nodes: improve multi-threading and avoid redundant work in CSV import
The position attribute was not initialized in parallel. Also there was an accidental
single threaded copy of each attribute because a `std::move` didn't work because
the variable was `const`.
2025-02-20 10:52:15 +01:00
Dalai Felinto
1584cd9aa5 Cleanup: Rename point cloud to pointcloud / POINT_CLOUD to POINTCLOUD
Though "Point Cloud" written as two words is technically correct and should be used in the UI, as one word it's typically easier to write and parse when reading. We had a mix of both before this patch, so better to unify this as well.

This commit also renames the editor/intern/ files to remove pointcloud_ prefix.
point_cloud was only preserved on the user facing strings:

* is_type_point_cloud
* use_new_point_cloud_type

Pull Request: https://projects.blender.org/blender/blender/pulls/134803
2025-02-19 17:11:08 +01:00
Jacques Lucke
3263dd6333 Fix: Geometry Nodes: unescape column names after csv import 2025-02-19 12:36:19 +01:00
Jacques Lucke
ac2cd6c1ef Geometry Nodes: make CSV parser more reliable and faster
This reimplements the CSV parser used by the (still experimental) Import CSV
node.

Reliability is improved by:
* Properly handling quoted fields.
* Unit tests.
* Generalizing the parser to be able to handle customized delimiter, quote and
  escape characters (those are not exposed in the node yet though).
* More accurate detection of column types by actually taking all values of a
  column into account instead of only the first row.

Performance is improved by designing the parser in a way that supports
multi-threaded parsing. I'm measuring about 5x performance improvement which
mainly comes from multi-threading. Some files I wanted to use for benchmarking
didn't load in the version that's in `main` but do load fine with this new
version.

The implementation is now split up into two parts:
1. A general CSV parser in `blenlib` that manages splitting a buffer into
   records and their fields.
2. Application specific parsing of fields into e.g. floats and integers which
   remains in `io/csv/importer`.

This separation simplifies unit testing and makes the core code more reusable.

Pull Request: https://projects.blender.org/blender/blender/pulls/134715
2025-02-19 11:10:59 +01:00
Hans Goudey
b62ba4af33 Fix: CSV import node ignores last character of some columns 2025-02-17 20:14:54 -05:00
Hans Goudey
7ab3087751 Cleanup: Remove unused CMake dependencies for CSV import 2025-02-10 12:23:50 -05:00
Hans Goudey
f01af5f972 Geometry Nodes: Remove extra copy step in CSV import node
Remove intermediate `CsvData` struct and create a point cloud
directly instead. Though the bottleneck is almost certainly parsing
the file, this removes a copy for the attribute values and reduces
peak memory usage.

Also do some small cleanups to the import process: use C++
casting, prefer StringRef over std::string, remove unnecessary
whitespace, and remove non-helpul comments.
2025-02-10 12:19:45 -05:00
Hans Goudey
78da0dda87 Cleanup: Warning in format string in CSV importer 2025-02-10 12:19:45 -05:00
Hans Goudey
ef8e85c436 Cleanup: Remove unnecessary indirection in CSV importer 2025-02-10 12:19:45 -05:00
Hans Goudey
7605382204 Cleanup: Include in CSV import header 2025-02-10 12:19:45 -05:00
Devashish Lal
1a62fdc82a Geometry Nodes: CSV import node
This commit implements a node to import CSV files as a point cloud.
The interface is minimal, with just a file path input. The type of each
column is chosen by whether the first value is an integer or a float
(those are currently the only supported types).

The goal of the node is to make it easier to get arbitrary data into
geometry nodes for visualization purposes, for example.

https://devtalk.blender.org/t/gsoc-2024-geometry-nodes-file-import-nodes/34482

Pull Request: https://projects.blender.org/blender/blender/pulls/126308
2025-02-10 16:56:52 +01:00