This function is used to retrieve the index of the n'th layer of
the specified type, if it exists. Currently the way it handles
an n value that is too big is inconsistent: if there are more
layers of different types following the ones of the correct type,
and n isn't too big, it will safely detect that the corresponding
layer is of a different type and return -1. However, if the value
results in a buffer overrun, it causes an assertion or crash.
This means that safe code should currently ensure n is always
correct, making the type check in this function pointless.
Moreover, checking the range in the caller may incur more
overhead than making this code safe.
This changes the assert into a check to ensure consistent behavior.
Pull Request: https://projects.blender.org/blender/blender/pulls/110812