Don't generate more verts than we need if we don't render vertex. Getting a big boost from 12fps to 20fps on my test scene