\mathrel conflict with \left,\right,\middle

Question

I refer to this question and particularly to this answer. I defined \set by

\usepackage{mathtools}
...
\DeclarePairedDelimiterX\set[2]{\{}{\}}{#1 \mathrel{\delimsize|} #2}

where \mathrel{\delimsize|} imitates \mid, which itself is of math relation type but not scaleable.

Although \set[\big]{...}{...}, \set[\Big]{...}{...} etc. work fine, the starred-form \set*{...}{...} does not. The problem is that, in this case, \delimsize expands to \middle and \mathrel{\middle|} leads to an error, which can easily be reproduced:

\left\{x \mathrel{\middle|} y\right\}

I could define \set alternatively like

\DeclarePairedDelimiterX\setnew[2]{\{}{\}}{#1 \;\delimsize|\; #2}

which should produce in most cases the same spaces around | as the version above. However, there is a difference if a set is in scriptstyle:

A_{\set{x}{y}}

puts no spaces around |, whereas

A_{\setnew{x}{y}}

still prints \;|\;.

Do you know a way to make \mathrel and \middle compatible?

Answer 1

No space is inserted in script or scriptscript styles if you precede \; with \nonscript:

\DeclarePairedDelimiterX\setnew[2]{\{}{\}}{%
  #1 \mathclose{}\nonscript\;\delimsize|\nonscript\;\mathopen{} #2%
}

Another solution that avoids \nonscript is

\DeclarePairedDelimiterX\set[2]{\{}{\}}
  {#1 \mathrel{}\mathclose{}\delimsize|\mathopen{}\mathrel{} #2}

Answer 2

If you are comfortable in using the alternative spacing you suggest:

\DeclarePairedDelimiterX\setnew[2]{\{}{\}}{#1 \;\delimsize|\; #2}

rather than

\DeclarePairedDelimiterX\set[2]{\{}{\}}{#1 \mathrel{\delimsize|} #2}

then try adding

\usepackage{etex}

in your document preamble. \middle is part of an etex extension.