Commit Graph

356 Commits

Author SHA1 Message Date
Campbell Barton
8a4ab1b3fe Fix assertion in BLI_path_normalize & add test 2023-05-19 19:25:08 +10:00
Campbell Barton
a5207ae24d Fix error returning the path length from BLI_path_normalize
Error in [0] caused the returned length to be wrong when skipping the
prefix. Also fix normalized path comparison failing on windows.

[0]: f45a985217
2023-05-17 17:47:35 +10:00
Campbell Barton
761eac2f5d Fix buffer overflow in BLI_path_append 2023-05-17 13:11:17 +10:00
Campbell Barton
0ae286be03 Fix trailing slashes causing normalized path comparison to fail 2023-05-17 13:11:17 +10:00
Sybren A. Stüvel
88f536dc02 Add always-positive modulo function
Add modulo function `mod_f_positive(f, n)` that returns a positive result,
regardless of the sign of `f`.

For example, `mod_f_positive(-0.1, 1.0)` returns `0.9`, whereas the
standard `fmodf()` function would return `-0.1`.

This is useful for rewrapping values to a specific interval.
2023-05-16 11:41:17 +02:00
Campbell Barton
13a3dfd788 BLI_path: remove FILE_MAX limit on BLI_path_frame{_range} functions
- Use BLI_str_replace_range to avoid a temporary string copy.
- Also add a buffer size argument to BLI_path_frame_range.
2023-05-15 09:09:08 +10:00
Campbell Barton
1d371706cf BLI_string: add BLI_str_replace_range for replacing a range
Add a convenient way to replace a range of text in the middle of a
string that make shrink or grow the string that handles corner cases
and keeps the string null terminated.
2023-05-15 09:08:59 +10:00
Campbell Barton
b1a426eac7 Cleanup: use string copy & printf macros 2023-05-13 17:34:21 +10:00
Bastien Montagne
3b647faec8 Fix (unreported) three wrong next/prev pointers order in ListBase link structs.
There was even one case in BLI ListBase tests!

Order was opposite of the one expected by the BLI ListBase code... Not
sure how, but this did not cause any issue apparently? But would expect
it to at least affect the order in which items in the list would be
iterated.
2023-05-12 19:02:39 +02:00
Sybren A. Stüvel
64f9a30cbd Test: actually use the right matrix for unit test
No functional changes, just a test working in debug mode too ;-)
2023-05-11 16:38:17 +02:00
Sybren A. Stüvel
39ba60e88d Math: add unit test for mat3_normalized_to_quat_fast
Add a unit test for the fix in 98334b8f7d

No functional changes.
2023-05-11 14:29:48 +02:00
Campbell Barton
cc9678b5fe Fix BLI_strncpy_utf8 over-read for code points containing a null byte
BLI_strncpy_utf8 didn't check for null bytes within bytes stepped
over by the variable length UTF8 encoding.

While a valid UTF8 string wont include these, it's possible Latin1
encoding or a truncated string includes such characters.

In this case, the entire string is copied as it's not the purpose of
this function to correct or strip invalid/truncated encoding,
only to prevent it from happening in the first place.
2023-05-11 11:44:38 +10:00
Campbell Barton
7f7804e62d Cleanup: use doxygen groups for BLI_string_tests 2023-05-09 20:32:13 +10:00
Campbell Barton
3958ae7241 Cleanup: use STRNCPY, SNPRINTF macros 2023-05-09 14:08:19 +10:00
Campbell Barton
1ab72e8459 Cleanup: use BLI_path_* prefix for path splitting functions
Also order string size after each string instead of grouping strings and
their sizes afterwards.
2023-05-02 21:08:13 +10:00
Campbell Barton
f154d37335 BLI_path: add a size argument to BLI_path_frame
Resolves potential buffer overflow in
USDVolumeWriter::construct_vdb_file_path which passed in a smaller
fixed size buffer than FILE_MAX.
2023-05-02 17:19:48 +10:00
Campbell Barton
6859bb6e67 Cleanup: format (with BraceWrapping::AfterControlStatement "MultiLine") 2023-05-02 09:37:49 +10:00
Jacques Lucke
7d38cef6d1 Geometry Nodes: use new BitGroupVector to find attribute propagation sets
A `BitGroupVector` is a compact data structure that allows storing multiple
bits per element, for example 5 bits per vertex. The implementation is
mostly just a wrapper around `BitVector`. There is some additional logic
to make sure that the bit span of every element is bounded (according
to the `is_bounded_span` function). This makes it more efficient to operate
on groups as a whole (e.g. `or` one group into another). In some sense,
this data structure can also be interpreted as a 2D bit array. Functions
like `append` can be added when they become necessary.

The new data structure is used to replace some `MultiValueMap` in
geometry nodes. This simplifies the code.
2023-04-28 16:54:43 +02:00
Hans Goudey
b87ccedd75 BLI: Add bit span operations and bounded bit spans
Most of this patch is by Jacques Lucke, from the simulation branch.

This commit adds generic expression evaluation for bit spans, helping
to generalize the optimizations that avoid processing a single bit
at a time. Operations like "for each 1 index", "or", and "and" are
already implemented in this pull request. Bits in full integers are
processed 64 at a time, then remaining bits are processed all at once.
The operations allow implementing a `copy_from` method for bit spans.

Currently this optimized evaluation is only implemented for simpler
bounded bit spans. Bounded bit spans have constraints on their bit
ranges that make them more efficient to process. Large spans must start
at the beginning of the first int, and small spans must start and end
within the first int.

Knowing these constraints at compile time reduces the number of edge
cases in the operations, but mainly allows skipping alignment between
multiple spans with different offsets.

Pull Request: https://projects.blender.org/blender/blender/pulls/107408
2023-04-28 15:43:34 +02:00
Campbell Barton
ff0cf45bc2 Fix potential buffer overflows from invalid string size arguments
- FILENAME_MAX was used when the output was later limited by FILE_MAX.
- Some were passing in incorrect/dummy sizes in a couple of places.
2023-04-28 21:49:05 +10:00
Campbell Barton
6701d24084 BLI_path: add string size arguments to sequence encode/decode functions 2023-04-28 21:33:38 +10:00
Jacques Lucke
bfcc2b1c4c BLI: add utility methods for serialization
This simplifies the code that works with the `BLI_serialize.hh` header.
The various `lookup` methods do a linear search. If there are only a
few elements that can even be faster than building the map first.
In the future it might be nice to transparently build and cache the
map internally if necessary.
2023-04-28 13:30:26 +02:00
Jacques Lucke
b4d914b676 BLI: support weak users and version in implicit sharing info
The main goal of these changes is to support checking if some data has
been changed over time. This is used by the WIP simulation nodes during
baking to detect which attributes have to be stored in every frame because
they have changed.

By using a combination of a weak user count and a version counter, it is
possible to detect that an attribute (or any data controlled by implicit
sharing) has not been changed with O(1) memory and time. It's still
possible that the data has been changed multiple times and is the same
in the end and beginning of course. That wouldn't be detected using this
mechanism.

The `ImplicitSharingInfo` struct has a new weak user count. A weak
reference is one that does not keep the referenced data alive, but makes sure
that the `ImplicitSharingInfo` itself is not deleted. If some piece of
data has one strong and multiple weak users, it is still mutable. If the
strong user count goes down to zero, the referenced data is freed.
Remaining weak users can check for this condition using `is_expired`.

This is a bit similar to `std::weak_ptr` but there is an important difference:
a weak user can not become a strong user while one can create a `shared_ptr`
from a `weak_ptr`. This restriction is necessary, because some code might
be changing the referenced data assuming that it is the only owner. If
another thread suddenly adds a new owner, the data would be shared again
and the first thread would not have been allowed to modify the data in
the first place.

There is also a new integer version counter in `ImplicitSharingInfo`.
It is incremented whenever some code wants to modify the referenced data.
Obviously, this can only be done when the data is not shared because then
it would be immutable. By comparing an old and new version number of the
same sharing info, one can check if the data has been modified. One has
to keep a weak reference to the sharing info together with the old version
number to ensure that the new sharing info is still the same as the old one.
Without this, it can happen that the sharing info was freed and a new
one was allocated at the same pointer address. Using a strong reference
for this purpose does not work, because then the data would never be
modified because it's shared.
2023-04-28 12:05:00 +02:00
Campbell Barton
3ad82ec5a4 Refactor: path normalize now collapses multiple '..' directories at once
- Avoid a separate memmove call for each `..`.
- Avoid ambiguous path stepping, where separator literals
  needed to be checked to avoid fence post errors.
- Correct & update the doc-string.
2023-04-28 12:23:29 +10:00
Hans Goudey
a6baf7beae BLI: Allow different integer types when filling span indices 2023-04-27 08:50:41 -04:00
Jacques Lucke
a57584e40d BLI: extract MapItem type to simplify iterating over map items 2023-04-25 12:15:02 +02:00
Campbell Barton
10fc2d6d96 Cleanup: remove basepath argument from BLI_path_normalize{_dir}
Keep these operations separate to simplify path handling logic & docs.
Many callers passed NULL and there were times paths were passed in which
didn't make any sense (where the paths had already been made absolute).
2023-04-24 12:23:04 +10:00
Campbell Barton
6ab225074d Cleanup: minor changes to BLI_path_normalize
- Don't attempt to skip the WIN32 drive component on relative paths
  it's possible this would have unexpected behavior but paths like
  are already unlikely to work as expected, e.g. "//C:\".
- Remove early returns for empty paths as there is no need for special
  handling.
- Add path normalize NOP tests
2023-04-24 11:58:47 +10:00
Campbell Barton
2ed6f738da BLI_path: support normalizing relative paths
Follow Python's os.path.normpath's handing on relative paths in more
situations.

- Leading './' is always stripped.
- Relative paths such as `//./a` & `///a` normalize to `//a`.
- Support for trailing `/..` on the end of paths.
- Avoid empty paths for relative paths that would otherwise
  resolve to an empty string, e.g:  `a/b/c/../../..` resolves to `.`.
2023-04-23 22:22:08 +10:00
Campbell Barton
651f9404a1 Cleanup: rename path_util 'Clean' tests to 'Normalize'
The function was renamed from `clean` but the tests weren't.
2023-04-23 14:30:52 +10:00
Campbell Barton
ac251cbe25 BLI_path: de-duplicate WIN32 logic for BLI_path_normalize
The only WIN32 specific behavior the drive or UNC prefix is skipped
(except for it's slash), allowing WIN32 & Unix paths to share the same
code-path.

Note that this changes behavior for WIN32 when there are too many parent
directories, where ".." that move before the root directory are removed
instead of being left in the path. e.g:
`C:\a\..\..\b` resolved to `C:\..\b` whereas now the result is `C:\b`.
This matches Python's `os.path.normpath`.
2023-04-23 14:30:51 +10:00
Campbell Barton
7884de02f3 Tests: add BLI_path_extension replace/ensure tests for overflow handling 2023-04-20 11:58:29 +10:00
Campbell Barton
95296dc3aa Cleanup: remove "Path" prefix from path_utils tests 2023-04-20 11:58:27 +10:00
Campbell Barton
9be0304b67 Cleanup: order expected value last in path_util tests 2023-04-20 11:58:25 +10:00
Campbell Barton
b6e527febb Tests: add more path extension tests 2023-04-20 11:58:23 +10:00
Campbell Barton
5294758830 Fix buffer overflow in BLI_path_frame_strip with long extensions
The file extension was copied into a buffer without checking it's size.
While large extensions aren't typical, some callers used small fixed
size buffers so an unusually named file could crash.
2023-04-20 11:47:22 +10:00
Campbell Barton
80edd10168 Fix regression in BLI_path_suffix for long extensions
Changes from [0] passed in a pointer size to BLI_strncpy.

[0]: f8e23e495b
2023-04-20 11:18:26 +10:00
Campbell Barton
86611a5fcc Tests: add tests for BLI_path_extension ensure & replace 2023-04-19 21:15:43 +10:00
Campbell Barton
f8e23e495b BLI_path: improve behavior of BLI_path_extension
Finding the extension included hidden files (starting with a '.'),
now finding the extension matches Python's `os.path.splitext` behavior
which has the advantate a hidden file is not considered one long
extension - with an empty name part.

Also update code to use BLI_path_extension in cases which previously
in-lined this logic.

BLI_path_frame_get path argument is now const,
it was being manipulated unnecessarily.
2023-04-19 11:33:26 +10:00
Campbell Barton
7f241fc773 Tests: add test for BLI_path_suffix & BLI_path_sequence_decode 2023-04-19 11:33:26 +10:00
Campbell Barton
b132118f89 Cleanup: balance doxygen grouping, minor grouping adjustment 2023-04-19 09:02:21 +10:00
Falk David
66158498de BLI: Return number of values removed from remove_if
Make the `remove_if` function for `Vector`, `VectorSet`, `Set`, and `Map` return the number of elements it removed.

Pull Request: https://projects.blender.org/blender/blender/pulls/107069
2023-04-18 13:28:14 +02:00
Clément Foucault
897a735151 BLI: Rotation: Add CartesianBasis transform_point and invert 2023-04-01 13:24:35 +02:00
Sergey Sharybin
a12a8a71bb Remove "All Rights Reserved" from Blender Foundation copyright code
The goal is to solve confusion of the "All rights reserved" for licensing
code under an open-source license.

The phrase "All rights reserved" comes from a historical convention that
required this phrase for the copyright protection to apply. This convention
is no longer relevant.

However, even though the phrase has no meaning in establishing the copyright
it has not lost meaning in terms of licensing.

This change makes it so code under the Blender Foundation copyright does
not use "all rights reserved". This is also how the GPL license itself
states how to apply it to the source code:

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) <year>  <name of author>

    This program is free software ...

This change does not change copyright notice in cases when the copyright
is dual (BF and an author), or just an author of the code. It also does
mot change copyright which is inherited from NaN Holding BV as it needs
some further investigation about what is the proper way to handle it.
2023-03-30 10:51:59 +02:00
Sergey Sharybin
d32d787f5f Clang-Format: Allow empty functions to be single-line
For example

```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```

becomes

```
OIIOOutputDriver::~OIIOOutputDriver() {}
```

Saves quite some vertical space, which is especially handy for
constructors.

Pull Request: https://projects.blender.org/blender/blender/pulls/105594
2023-03-29 16:50:54 +02:00
Jacques Lucke
d81d743537 BLI: support == and != for Set
This makes it more convenient to check if two sets contain the same keys.
2023-03-28 14:16:57 +02:00
Jacques Lucke
fbcddfcd68 BLI: add core types for supporting implicit-sharing
The overall goal is to use implicit-sharing in many places in Blender
that currently do unnecessary copies. See #95845 for more details.

This commit only adds the base data structures in blenlib and uses those
in `GeometrySet` and `AnonymousAttributeID`, which used a more ad-hoc
version of implicit sharing already. #105994 lists some more places where
support for implicit sharing can be added (most notably: custom data layers).

Pull Request: https://projects.blender.org/blender/blender/pulls/105994
2023-03-28 13:57:51 +02:00
Campbell Barton
bb2dc141f2 Cleanup: spelling in comments 2023-03-27 12:08:14 +11:00
Sergey Sharybin
08466a93e3 Fix security warning generated by std::tmpnam
Effectively replicate the behavior of the function in the manner
which is used for autosave file.

There might be better a solution which is cross-platform and does
not suffer from the time of check, time of use (TOCTOU) vector of
attack. This seems to be a bigger project to figure out, so until
then silence the warning: it is fine since the directory is only
used to chdir to, so worst case an external attacker can introduce
is a test failure.
2023-03-24 14:52:43 +01:00
Sergey Sharybin
8c38d29feb BLI: Add utility to access system-wide temporary directory
It provides path to a directory suitable for storing temporary files
which satisfies the following conditions:

- The directory exists.
- The return path has trailing directory separator.

It is based on the code used in the appdir.c to get the temporary
directory.

For the C++ people: this is similar to the temp_directory_path()
from the std::filesystem.

No functional changes expected as it is a new code which is planned
to be used on other places as a followup development.
2023-03-24 14:52:43 +01:00