Prepare the code to more easily support pointers remapping for tracking objects. Should be no functional changes.