A very Bad Bug!
On every draw-object, a function free_old_images() was called which
was freeing "unused" images during renders/bakes
This was a left-over from 2.4x code, missed it altogether.
I'm sure this fix will solve a lot of render crashing... :)