Trust user count to actually delete or not the dragged ID when current
dragging is cancelled, since it may be already used by others.
NOTE: This is more a band-aid fix than anything else, cancelling drag
has a lot of other issues here (like never deleting any indirectly
linked/appended data, etc.). It needs a proper rethink in general.
66328db703