This was due to the AO random sampling using the same "seed" as the AA jitter. Decorelating the noise fixes the issue.