When baking relative to multires level 0, it would apply one level of simple subdivision still, which gave artifacts.