There were two issues:
1. Memory leak: std:;erase does not call delete on the
pointer (which is actually a good idea),
2. After MIS was disabled in viewport render there was
no way to bring MIS back.
Now instead of removing light from the scene data we
kind of tagging it for an ignore. Possible cleanup
would be to add Light::is_enabled and use that instead
of passing weird and wonderful function arguments.