This solution is causing the modifier regression test to fail. Also fix a mistake to allocate `vert_dest_map` twice.