As pointed out by @pablovazquez the selection behavior
of the face loop selection in paint modes was missing the toggle feature.
The logic goes like this:
If any of the 2 adjacent faces to the selected edge are selected,
deselect instead of select.
There is still an inconsistency which I left in for this patch,
but is up to discussion.
CTRL+ALT+SHIFT click does an actual deselect in paint mode.
In edit mode it does toggle as well, just as ALT+SHIFT click
I can remove that feature if it isn't wanted
Pull Request: https://projects.blender.org/blender/blender/pulls/108753