thmtools notebraces bug?

Question

\documentclass{article}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{thmtools}

\declaretheoremstyle[notebraces={[}{]}]{style}
\declaretheorem[style=style]{theorem}

\begin{document}

\begin{theorem}[Pythagorean Theorem (Pythagoras)]
\end{theorem}

\end{document}

results in

Theorem 1 [Pythagorean Theorem (Pythagoras]).

but really should be

Theorem 1 [Pythagorean Theorem (Pythagoras)].

Answer 1

Option notebraces replaces the parentheses with the given symbols with the help of macro \thmt@embrace:

\def\thmt@embrace#1#2(#3){#1#3#2}

Argument #1 and #2 are the new open and closing braces, \thmt@embrace is fed with the full title inclusive standard parentheses, which are detected by the parameter text (#3). However, in this case, #3 already contains ( and ), thus that the detection of the correct closing ) fails.

Workaround: Protect the inner parentheses by curly braces, as already suggested by Barbara Beeton's comment:

\begin{theorem}[Pythagorean Theorem {(Pythagoras)}]
\end{theorem}

The curly braces should be at least partially inside the title as in the example above to avoid automatic removal of the curly braces, when the title argument is read.

Another way is fixing \thmthead@plain to insert the protecting curly braces internally:

\documentclass{article}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{thmtools}

\makeatletter
\renewcommand*{\thmhead@plain}[3]{%
  \thmname{#1}%
  \thmnumber{%
    \@ifnotempty{#1}{ }%
    \@upn{#2}%
  }%
  \thmnote{ {\the\thm@notefont({#3})}}%
}
\makeatother

\declaretheoremstyle[notebraces={[}{]}]{style}
\declaretheorem[style=style]{theorem}

\begin{document}
  \begin{theorem}[Pythagorean Theorem (Pythagoras)]
  \end{theorem}
\end{document}

Answer 2

The file amsthm.sty has explicit (...) for the note and thmtools.sty (better, thm-amsthm.sty) relies on this.

I suggest changing the delimiters, rather than bracing them, because also \NOTE uses it and only changing \thmhead@plain is not sufficient.

\documentclass{article}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{thmtools}

\makeatletter
%%% from amsthm.sty
\def\thmhead@plain#1#2#3{%
  \thmname{#1}\thmnumber{\@ifnotempty{#1}{ }\@upn{#2}}%
%%% the line below had (##3)
  \thmnote{ {\the\thm@notefont\thm@lparen#3\thm@rparen}}}

%%% from thm-amsthm.sty
\def\thmt@setheadstyle#1{%
  \thmt@style@headstyle{%
    \def\NAME{\the\thm@headfont ##1}%
    \def\NUMBER{\bgroup\@upn{##2}\egroup}%
%%% the line below had (##3)
    \def\NOTE{\if=##3=\else\bgroup\thmt@space\the\thm@notefont\thm@lparen##3\thm@rparen\egroup\fi}%
  }%
  \def\thmt@tmp{#1}%
  \@onelevel@sanitize\thmt@tmp
  %\tracingall
  \ifcsname thmt@headstyle@\thmt@tmp\endcsname
    \thmt@style@headstyle\@xa{%
      \the\thmt@style@headstyle
      \csname thmt@headstyle@#1\endcsname
    }%
  \else
    \thmt@style@headstyle\@xa{%
      \the\thmt@style@headstyle
      #1%
    }%
  \fi
  %\showthe\thmt@style@headstyle
}
%%% the line below had (#3)
\def\thmt@embrace#1#2\thm@lparen#3\thm@rparen{#1#3#2}
%%% added for default
\def\thm@lparen{(}\def\thm@rparen{)}
\makeatother

\declaretheoremstyle[notebraces={[}{]}]{style}
\declaretheorem[style=style]{theorem}

\begin{document}

\begin{theorem}[Pythagorean Theorem (Pythagoras)]
\end{theorem}

\end{document}