Part of #118145. There is more cleanup possible here, particularly doing some loop fission and making the brush use our common utilities will help remove some code and make better use of auto-vectorization. But with this commit the code we want to remove is no longer used, so we can stop here for now.