The problem here was that the sync code overwrites the point data, but didn't reset the transform_applied flag, so the rest of the code assumed that the points are already in world space and didn't apply the transform again.