The first optional argument of \documentclass
takes a comma separated list of options.
So:
\documentclass[11pt,reqno]{amsbook}
\begin{document}
x
\end{document}
If you enter
\documentclass[11pt][reqno]{amsbook}
\begin{document}
x
\end{document}
Then essentially LaTeX sees
\documentclass[11pt]{}[
and fails to find a class file with empty name.
With an old release (2019 or older) you get the error
LaTeX Error: File `.cls' not found.
With a current release, you get several internal errors as the new filehooks code fails on the empty filename.
! LaTeX hooks Error: Missing (empty) default label on line 1.
(hooks) This command was ignored.
Type <return> to continue.
...
l.1 \documentclass[11pt][r
eqno]{amsbook}
?
\documentclass[11pt,reqno]{amsbook}
instead of\documentclass[11pt][reqno]{amsbook}
.\documentclass
must all be enclosed in the same set of brackets[ ]
and separated by commas. (The options you wish to use are both defined; it's the input format that's not correct.)[11pt]
should have said what the syntax was, so the question is more where have you seen any suggestion that two options are possible there, we could get that documention fixed.