Before we always did a float conversion of the colors even if we
wouldn't use the result.
Because the float conversion does alpha premultiplication, we can
use some shortcuts to skip a lot of computations
In a edit with a lot of fully transparent pixels, I observed a reduction
in total render time from 1m 31s to 1m 17s.
Pull Request: https://projects.blender.org/blender/blender/pulls/117134