Design of new brackets

There are many types of brackets (), [], {}, ||, <>, etc. In the context I use them all have already a fixed mathematical meaning; therefore I need another one.

The brackets I use have two floors, so () looks like the binomial coefficient or {} like Knuth's notation for the Stirling set numbers, etc.

The new pair of brackets which I would like to create is also a double-decker and looks (with arguments n and k) like

(n)
(k)


However there is no gap between the upper and the lower part. On the left hand side the lowest point of the upper arc ( should touch the highest point of the lower arc (. This design mirrored gives the symbol on the right hand side.

Is there any hope to implement these two brackets with the tools of (La)TeX?

Here the symbol created with a paint program. For artists: A connotation with epsilon is welcome.

Edit: After seeing the first two proposals I notice an issue: the height of the double-decker should match the height of the binomial (in fact it is to be expected that they often will stand side by side). Manuel's double-decker is a bit too large in the in-line case and too small in the display case. Hagen's is not stable in the display mode.

The two solutions side by side:

Second edit:

The answer of Gustavo is great but the answer of Manuel is much simpler and therefore will be preferred under some circumstances. As Gustavo points out the command has to be protected inside moving arguments. Here some test code:

\documentclass{scrartcl}
\usepackage{mathtools}
\usepackage{scalerel}
\usepackage{xparse}
\NewDocumentCommand{\stirling}{omm}
{%
\genfrac\{\}{0pt}{}{#2}{#3}%
\IfValueT{#1}{_{\!#1}}%
}
\makeatletter
\newcommand*\doubledecker[2]
{\mathpalette{\doubledeckeraux{#1}{#2}}\relax}
\newcommand*\doubledeckeraux[3]
{\vcenter{\offinterlineskip
\ialign{$\ifx\textstyle#3\scriptstyle \else\ifx\scriptstyle#3\scriptscriptstyle \fi\fi \ifx\displaystyle#3\expandafter\bigl\fi( \hfil##\hfil \ifx\displaystyle#3\expandafter\bigl\fi)\m@th$\cr
\smash{#1}\cr
\smash{#2}\cr}}}
\makeatother
\begin{document}
This is how doubledecker looks inline: $\doubledecker{n}{k}$
${\doubledecker{n}{k}}_m \ne \stirling[m]{n}{k}$

It needs another pair of $\{\}$ to display the index $m$ properly.

Here doubledecker has to be protected:
\begin{table}[h]\centering
\begin{tabular}[c]{c} $x$  \end{tabular}
\caption{ ${\protect \doubledecker{n}{k}}_m$ }
\end{table}
\end{document}

• Interesting. Hint: provide a picture of what you want, hand-drawn or with paint or Inkscape or gimp. Having a visual aid is good :-) – Johannes_B Jul 4 '15 at 10:02
• IIRC the growable \{\} is produced by gluing together five parts (upper tip, line, middle part, line, lower tip). If one switches the middle part among left and right, this might just be what is desired. But I'm afraid that will leave out a few small sizes where the braces are not glued from parts ... – Hagen von Eitzen Jul 4 '15 at 14:10
• Hm, can you elaborate what''s "not stable"? The test image I posted actually consisted only of display mode (and it does work in inline mode as well, but there it might be a bit too large overall because it uses at least two strtheights ...) – Hagen von Eitzen Jul 4 '15 at 20:34
• Hagen, as my picture shows the two arcs do not meet in the display mode in your solution. (It's the third symbol in the second line.) If there is a difference between your output and mine than I cannot explain that. (Perhaps the sum sign makes a difference?) – Sophia Antipolis Jul 4 '15 at 22:03
• Here is my test code: pastebin.com/JbQhcdAc – Sophia Antipolis Jul 4 '15 at 22:43

In the past few days, I have been caught by a sudden passion for virtual fonts, and I feel that, without mention of a solution that employs them, the discussion on this question is somewhat incomplete. After all, this is almost surely what Hagen von Eitzen had originally in mind (see other answers). Of course, the solutions that have already been presented are much more direct and easier to implement, while providing excellent results, so this one should be regarded as a kind of collateral (and pedantic) comment, made just for the sake of exhaustiveness.

Showcase

So that you immediately get the flavor of the result we are going to obtain, I’ll show straight away a couple of sample pages. The first attempt I made was along the lines suggested by Hagen, that is, to invert the middle segment of braces. Of course, this only works from a certain size up; for smaller sizes, I was building the required symbols by stacking two round parentheses. Here is what I got:

Clearly unsatisfactory: if the symbols have to look like a pair of “double-decker” parentheses, they should do so in all sizes. So, I thought of a different solution, that is, to build the extensible parentheses from four pieces: the top, bottom, and repeatable pieces, which are identical to their counterparts used in (extensible) round parentheses, plus a middle piece manufactured stacking the bottom piece of a round parenthesis over the corresponding top piece. I also made some adjustment to the \big and \Big characters, borrowing the parentheses from cmr6 and cmr9, respectively, rather than scaling down one of the parentheses of cmex10 itself, and here is the result:

Much better! But, you will remark, I have also taken one step that seems to go exactly in the opposite direction: in the normal size, the stacked parentheses have been replaced by the left and right “quill” symbols (from the TS1, or “text companion”, encoding), which look just like braces with the central tip reversed. The reason for which I did so is that I wanted to reproduce, as far as possible, the behavior of the standard symbol fonts: while there is only one version of the “math extension” font, namely, cmex10, the “small” symbols are available in a variety of typefaces and sizes. Using the tc fonts, which are widely available in dozens of different sizes and faces, I get the same variety “for free”, without defining dozens of virtual fonts.

[Note: the AMS font collection also includes “math extension” fonts in smaller sizes, namely, cmex7, cmex8, and cmex9. The process, described below, from which we derive our ddeckparex10 virtual font from cmex10 can be applied, with obvious adaptations, to these fonts as well, obtaining analogous virtual fonts that could be called ddeckparex7, ddeckparex8, and ddeckparex9. You could then use these virtual fonts in a way similar to the way in which the package amsfonts uses the additional AMS fonts. We’ll return on this point below.]

Defining the virtual font

Please remember that there is already an answer that describes this process in detail. This allows me to keep the following comments relatively terse.

In this case, the virtual font must be defined essentially from scratch (although substantial excerpts can be copied from cmex10.pl). How do you do this? Well, it’s not much different from what you usually do with computers: you write a source file in a human-oriented programming language, and then you have a compiler translate it into a machine-oriented form. In this case, the “source file” is called a Virtual Property List file, and the “compiler” is the vptovf program (for “Virtual Property-list TO Virtual Font”). To learn how to write a Virtual Property List file (with extension .vpl), you can, for example, type texdoc vptovf at your terminal prompt and read sections 7–19 of the document that is displayed. Or, if you don’t care, you could just save the following code into a file called ddeckparex10.vpl and placed in a working directory of your choice:

(FAMILY DDKPAREX)
(CODINGSCHEME FONTSPECIFIC)
(DESIGNSIZE R 10.0)
(COMMENT DESIGNSIZE IS IN POINTS)
(COMMENT OTHER SIZES ARE MULTIPLES OF DESIGNSIZE)
(MAPFONT D 0 (FONTNAME cmex10))
(MAPFONT D 1 (FONTNAME cmr6)  (FONTAT R .6)(FONTDSIZE R 6.0))
(MAPFONT D 2 (FONTNAME cmr9)  (FONTAT R .9)(FONTDSIZE R 9.0))
(MAPFONT D 3 (FONTNAME cmex10)(FONTAT R .8333333))
(FONTDIMEN
(SLANT R 0.0)
(SPACE R 0.0)
(STRETCH R 0.0)
(SHRINK R 0.0)
(XHEIGHT R 0.430555)
(EXTRASPACE R 0.0)
)
(CHARACTER H 10
(CHARWD R 0.4333284)
(CHARHT R 0.039999)
(CHARDP R 1.160013)
(NEXTLARGER H 12)
(MAP
(SELECTFONT D 1)
(MOVERIGHT R 0.0481476)(MOVEDOWN R 0.410001)
(PUSH)(SETCHAR O 50)(POP)
(MOVEDOWN R 0.600006)
(SETCHAR O 50)
)
)
(CHARACTER H 11
(CHARWD R 0.4333284)
(CHARHT R 0.039999)
(CHARDP R 1.160013)
(NEXTLARGER H 13)
(MAP
(SELECTFONT D 1)
(MOVERIGHT R 0.0962952)(MOVEDOWN R 0.410001)
(PUSH)(SETCHAR O 51)(POP)
(MOVEDOWN R 0.600006)
(SETCHAR O 51)
)
)
(CHARACTER H 12
(CHARWD R 0.41967345)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
(NEXTLARGER H 14)
(MAP
(SELECTFONT D 2)
(MOVERIGHT R 0.01998445)(MOVEDOWN R 0.6710001)
(PUSH)(SETCHAR O 50)(POP)
(MOVEDOWN R 0.900009)
(SETCHAR O 50)
)
)
(CHARACTER H 13
(CHARWD R 0.41967345)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
(NEXTLARGER H 15)
(MAP
(SELECTFONT D 2)
(MOVERIGHT R 0.0399689)(MOVEDOWN R 0.6710001)
(PUSH)(SETCHAR O 51)(POP)
(MOVEDOWN R 0.900009)
(SETCHAR O 51)
)
)
(CHARACTER H 14
(CHARWD R 0.458336)
(CHARHT R 0.039999)
(CHARDP R 2.360025)
(NEXTLARGER H 16)
(MAP
(PUSH)(SETCHAR O 0)(POP)
(MOVEDOWN R 1.200012)
(SETCHAR O 0)
)
)
(CHARACTER H 15
(CHARWD R 0.458336)
(CHARHT R 0.039999)
(CHARDP R 2.360025)
(NEXTLARGER H 17)
(MAP
(PUSH)(SETCHAR O 1)(POP)
(MOVEDOWN R 1.200012)
(SETCHAR O 1)
)
)
(CHARACTER H 16
(CHARWD R 0.597224)
(CHARHT R 0.039999)
(CHARDP R 2.9600315)
(NEXTLARGER H 18)
(MAP
(SELECTFONT D 3)
(MOVERIGHT R 0.0497687)(MOVEUP R 0.0199995)
(PUSH)(SETCHAR O 20)(POP)
(MOVEDOWN R 1.50001525)
(SETCHAR O 20)
)
)
(CHARACTER H 17
(CHARWD R 0.597224)
(CHARHT R 0.039999)
(CHARDP R 2.9600315)
(NEXTLARGER H 19)
(MAP
(SELECTFONT D 3)
(MOVERIGHT R 0.0497687)(MOVEUP R 0.0199995)
(PUSH)(SETCHAR O 21)(POP)
(MOVEDOWN R 1.50001525)
(SETCHAR O 21)
)
)
(CHARACTER H 18
(CHARWD R 0.597224)
(CHARHT R 0.039999)
(CHARDP R 3.560037)
(NEXTLARGER H 1A)
(MAP
(PUSH)(SETCHAR O 20)(POP)
(MOVEDOWN R 1.800018)
(SETCHAR O 20)
)
)
(CHARACTER H 19
(CHARWD R 0.597224)
(CHARHT R 0.039999)
(CHARDP R 3.560037)
(NEXTLARGER H 1B)
(MAP
(PUSH)(SETCHAR O 21)(POP)
(MOVEDOWN R 1.800018)
(SETCHAR O 21)
)
)
(CHARACTER H 1A
(CHARWD R 0.7361145)
(CHARHT R 0.039999)
(CHARDP R 4.760049)
(NEXTLARGER H 1C)
(MAP
(PUSH)(SETCHAR O 22)(POP)
(MOVEDOWN R 2.400024)
(SETCHAR O 22)
)
)
(CHARACTER H 1B
(CHARWD R 0.7361145)
(CHARHT R 0.039999)
(CHARDP R 4.760049)
(NEXTLARGER H 1D)
(MAP
(PUSH)(SETCHAR O 23)(POP)
(MOVEDOWN R 2.400024)
(SETCHAR O 23)
)
)
(CHARACTER H 1C
(CHARWD R 0.79167)
(CHARHT R 0.039999)
(CHARDP R 5.960062)
(NEXTLARGER H 30)
(MAP
(PUSH)(SETCHAR O 40)(POP)
(MOVEDOWN R 3.00003)
(SETCHAR O 40)
)
)
(CHARACTER H 1D
(CHARWD R 0.79167)
(CHARHT R 0.039999)
(CHARDP R 5.960062)
(NEXTLARGER H 31)
(MAP
(PUSH)(SETCHAR O 41)(POP)
(MOVEDOWN R 3.00003)
(SETCHAR O 41)
)
)
(CHARACTER H 30
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
(VARCHAR
(TOP H 30)
(MID H 38)
(BOT H 40)
(REP H 42)
)
)
(CHARACTER H 31
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
(VARCHAR
(TOP H 31)
(MID H 39)
(BOT H 41)
(REP H 43)
)
)
(CHARACTER H 38
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 3.560037)
(MAP
(PUSH)(SETCHAR H 40)(POP)
(MOVEDOWN R 1.800018)
(SETCHAR H 30)
)
)
(CHARACTER H 39
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 3.560037)
(MAP
(PUSH)(SETCHAR H 41)(POP)
(MOVEDOWN R 1.800018)
(SETCHAR H 31)
)
)
(CHARACTER H 40
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
)
(CHARACTER H 41
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
)
(CHARACTER H 42
(CHARWD R 0.875003)
(CHARDP R 0.600006)
)
(CHARACTER H 43
(CHARWD R 0.875003)
(CHARDP R 0.600006)
)


After this, “compile” the virtual font by typing

vptovf ddeckparex10.vpl


at the command line prompt; this will generate, always in that same directory, two machine-oriented files called ddeckparex10.tfm and ddeckparex10.vf, respectively, from which (pdf)TeX gets all the necessary information to typeset your new “double-decker” parentheses. More precisely, TeX in itself only needs the .tfm file to do its typesetting job, while the .vf file is used either by a DVI previewer (or printer driver) to learn how to display (or print) the new symbols, or by pdfTeX to include in the generated PDF file appropriate commands that reconstruct the new glyphs starting exclusively from the CMEX10, CMR6, and CMR9 fonts: only these fonts will be actually included in the PDF. The same principle applies to most DVI-to-something converters: for example, dvips will generate a PostScript file containing only references to the three aforementioned fonts (plus all the other fonts used by the document, of course).

The files ddeckparex10.tfm and ddeckparex10.vf must be saved in a place where (pdf)TeX (or dvips, etc.) can find them. You have essentially three possibilities:

1. Place both of them in the same directory as the .tex file you want to compile. This is a quick and easy solution, but, of course, it cannot be applied in the long term.

2. Place them in your personal texmf tree. How to do this varies from system to system. For example, for my MacTeX installation this means:

• place the file ddeckparex10.tfm somewhere under ~/Library/texmf/fonts/tfm/ (that is, either directly inside that directory, or inside a subdirectory thereof—or sub-sub-…-sub-directory);

• place the file ddeckparex10.vf somewhere under ~/Library/texmf/fonts/vf/ (as above).

Note that you might have to create the necessary directories under ~/Library/.

3. Place them in your local machine-wide texmf tree. Of course, in order to do so you need appropriate access priviledges. This too varies from system to system; for me it means:

• place the file ddeckparex10.tfm somewhere under /usr/local/texlive/texmf-local/fonts/tfm/;

• place the file ddeckparex10.vf somewhere under /usr/local/texlive/texmf-local/fonts/vf/.

At least with TeXLive, after this you have to issue the command mktexlsr from the command line, in order to index your changes.

Whatever way you have chosen, you may now run

pdflatex nfssfont


to produce the character table of the new ddeckparex10 font; here it is:

In this way, you can also check that the virtual font is installed properly. By the way, this was the character table of my first attempt:

A package that supports the virtual font

The following code

\NeedsTeXFormat{LaTeX2e}\relax % LaTeX2e is required!
\ProvidesPackage{ddkparex}
[2015/07/15 v0.01 (**development**)]

\@ifdefinable\@ddkp@saved@encoding{}
\let\@ddkp@saved@encoding=\encodingdefault
\RequirePackage[TS1]{fontenc}
\let\encodingdefault=\@ddkp@saved@encoding
\fontencoding\encodingdefault\selectfont

\DeclareFontFamily{U}{ddkparex}{\hyphenchar\font \m@ne}
\DeclareFontShape{U}{ddkparex}{m}{n}{<->ddeckparex10}{}

\DeclareSymbolFont{doubleDeckerParenthesesSmall}{TS1}{cmr}     {m} {n}
\DeclareSymbolFont{doubleDeckerParenthesesLarge}{U}  {ddkparex}{m} {n}
\SetSymbolFont{doubleDeckerParenthesesSmall}{bold}{TS1}{cmr}{b}{n}
\DeclareMathDelimiter{\lddparen}{\mathopen}
{doubleDeckerParenthesesSmall}{"A0}
{doubleDeckerParenthesesLarge}{"10}
\DeclareMathDelimiter{\rddparen}{\mathclose}
{doubleDeckerParenthesesSmall}{"A1}
{doubleDeckerParenthesesLarge}{"11}


implements a simple package, called ddkparex, that defines two new commands:

• \lddparen produces a left (opening) “double-decker” parenthesis;

• \rddparen produces a right (closing) “double-decker” parenthesis.

These two commands can be used in exactly the same way as other delimiter-producing commands like \{, \rangle, etc., either by themselves or with \left, \right, \biggl, \Biggr, \genfrac, … It can’t be easier.

The above code should be saved, of course, in a file named ddkparex.sty, placed in some valid TeX input directory, that, in the simplest case, could be the same directory containing the .tex file you want to compile.

Testing the new commands

The following compilable example will generate the second of the sample pages that were shown in the initial “showcase”:

\documentclass[a4paper]{article}
\usepackage[T1]{fontenc}
\usepackage{amsmath}
\usepackage{ddkparex}

\begin{document}

First, let's have a look to the delimiters set in predetermined sizes.
The first two sizes, normal'' and \verb|\big|,
and are exactly the same size of the other delimiters:
$$(x) \ne \lddparen x\rddparen \simeq \bigl\lddparen x \bigr\rddparen \ne \bigl(x\bigr)$$;
and $$\bigl\lddparen\!\tfrac{n}{k}\!\bigr\rddparen$$ too.

Here is a display with \verb|\Big|, \verb|\bigg|, and \verb|\Bigg| delimiters,
in that order:
\begin{align*}
\Bigl\lddparen  \frac{p}{b} \Bigr\rddparen
&\neq \Bigl(    \frac{p}{b} \Bigr)
&
\biggl\lddparen \frac{p}{b} \biggr\rddparen
&\neq \biggl(   \frac{p}{b} \biggr)
&
\Biggl\lddparen \frac{p}{b} \Biggr\rddparen
&\neq \Biggl(   \frac{p}{b} \Biggr)
\end{align*}
Each variant is compared with a similarly-sized ordinary parenthesis.  Note that
we have also included the fraction line to demonstrate that the delimiters are
vertically centered.

But delimiters of even \verb|\biggg|-er size are available too; they can be
accessed via \verb|\left| and \verb|\right|:
\begin{align*}
\left\lddparen
\begin{matrix}
1 & 0 & 1 & 0  \\
0 & 1 & 0 & 1  \\
1 & 0 & 1 & 0  \\
0 & 1 & 0 & 1  \\
\end{matrix}
\right\rddparen
&&
\left\lddparen
\begin{matrix}
1 & 0 & 1 & 0 & 1  \\
0 & 1 & 0 & 1 & 0  \\
1 & 0 & 1 & 0 & 1  \\
0 & 1 & 0 & 1 & 0  \\
1 & 0 & 1 & 0 & 1  \\
\end{matrix}
\right\rddparen
\end{align*}
And here are the extensible ones:
\begin{align*}
\left\lddparen
\begin{matrix}
1 & 2 & 3 & 4 & 5 & 6  \\
2 & 0 & 0 & 0 & 0 & 0  \\
3 & 0 & 0 & 0 & 0 & 0  \\
4 & 0 & 0 & 0 & 0 & 0  \\
5 & 0 & 0 & 0 & 0 & 0  \\
6 & 0 & 0 & 0 & 0 & 0  \\
\end{matrix}
\right\rddparen
&&
\left\lddparen
\begin{matrix}
1 & 2 & 3 & 4 & 5 & 6 & 7 & 8  \\
2 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
3 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
4 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
5 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
6 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
7 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
8 & 0 & 0 & 0 & 0 & 0 & 0 & 0  \\
\end{matrix}
\right\rddparen
&&
\left\lddparen
\begin{matrix}
1\\2\\3\\4\\5\\6\\7\\8\\9\\0
\end{matrix}
\right\rddparen
\end{align*}
That's \emph{much} better!

Now, what about the quill'' symbols of the TS1~encoding, that the
\textsf{textcomp} package makes available with the commands \verb|\textlquill|
and \verb|\textrquill|?  Using them, as we do, for the normal size of the
double decker'' parentheses, the dissimilarity between the symbols in this
size and in the \verb|\big| one is evident; I'll show the two pairs again,
together with other kinds of parentheses as well:
$$\{x\} \neq \lddparen x\rddparen \simeq \bigl\lddparen x\bigr\rddparen \neq \bigl(x\bigr) \simeq (x)$$.
But the advantage is that, doing so, we have the symbol available in dozen of
size and face variants for free'': see, for example, the next two paragraphs.

{\Large\bfseries\boldmath A bolded formula:
$$(x) \ne \lddparen x\rddparen$$.\par}
{\Large\bfseries The same formula in medium weight:
$$(x) \ne \lddparen x\rddparen$$.\par}

In order to replicate this with a symbol constructed by us, we would need to
build dozens of virtual fonts (each containing only two characters).

{\Large\bfseries\boldmath But $$(x) \ne \lddparen x\rddparen \ne \bigl\lddparen x\bigr\rddparen \ne \bigl(x\bigr)$$.\par}

Indeed, the \texttt{cmex10} font hasn't got a bold variant.

\end{document}


By looking at this code, you can see that the two new commands are used exactly like any other delimiter-producing command, as said above.

How is the virtual font defined?

Let’s now examine the ddeckparex10.vpl file, to see what has been put into it and how the new characters have been defined. Note: This section is pretty long, and is aimed only to interested readers.

You will find it easier to follow the forthcoming discussion if you can consult the description of the metrics of the cmex10 font: a human-oriented form of this description (that is, a Property List, or .pl, file for that font) can be produced by typing

tftopl cmex10.tfm cmex10.pl


at the command line.

The opening lines

(FAMILY DDKPAREX)
(CODINGSCHEME FONTSPECIFIC)
(DESIGNSIZE R 10.0)
(COMMENT DESIGNSIZE IS IN POINTS)
(COMMENT OTHER SIZES ARE MULTIPLES OF DESIGNSIZE)
(MAPFONT D 0 (FONTNAME cmex10))
(MAPFONT D 1 (FONTNAME cmr6)  (FONTAT R .6)(FONTDSIZE R 6.0))
(MAPFONT D 2 (FONTNAME cmr9)  (FONTAT R .9)(FONTDSIZE R 9.0))
(MAPFONT D 3 (FONTNAME cmex10)(FONTAT R .8333333))


are plain. First of all, we say that this file defines a font family which is linked to the conventional name DDKPAREX; this name is not cared for by the texmf software, and actually, in our context, serves no purpose at all. At any rate, we also indicate that this font adhere to no predefined font encoding scheme. On the contrary, the specification of the design size (10.0 printers’ point) is important, since almost all other sizes given in the file are relative to the design size. Note, however, that if the font is loaded at a different size (which may well happen), the subsequent relative sizes will be multiplied by the size at which the font is loaded, not by the design size, thereby providing for appropriate scaling; for example, if you typeset a document with the 12pt option, the font will be loaded at 12pt, and all character dimensions, as well as the dimensions given in the commands that manifacture the characters themselves, will refer to absolute dimensions that are 1.2 times larger than the ones used with the 10pt option.

The subsequent lines declare the fonts that the characters used to build our virtual font are taken from, and assign numbers to them which will identify these fonts in the sequel.

Note that, although no \fondimen parameter at all is needed for this font, we do define the seven standard ones, just in case I’m forgetting something. The values are simply copied from cmex10.pl.

Let us now look at the definition of a typical character:

(CHARACTER H 14
(CHARWD R 0.458336)
(CHARHT R 0.039999)
(CHARDP R 2.360025)
(NEXTLARGER H 16)
(MAP
(PUSH)(SETCHAR O 0)(POP)
(MOVEDOWN R 1.200012)
(SETCHAR O 0)
)
)


This is the left “double-decker” parenthesis at \bigg size, which is built stacking two \big left parentheses taken from cmex10, that is, our local font number 0. The width of this character is copied from the width of the \big (left) parenthesis in cmex10, while the height and the depth are simply those of the \bigg parentheses, always in cmex10; the height-plus-depth equals 2.400024 relative units, that is, exactly twice as much as the relative height-plus-depth of the \big parentheses.

Then, we declare that this character has a successor in this font, which turns out to be the left “double-decker” parenthesis in \Bigg size. You can check that it also has a predecessor, corresponding to the \Big size.

The instructions for building the virtual character follow next: this is where our construction actually takes place. Since the height of our virtual character turns out to be exactly equal to that of its topmost piece, we can simply typeset this piece at the current reference point (it is character 0 from local font 0, that is, cmex10 unscaled; it is not necessary to specify the font number as 0 is implied by default). After this, we return at the reference point ((PUSH)/(POP)), move down by half the height-plus-depth, and typeset another identical character: that’s all.

Other characters need a definition that is slightly more involved; for example, this is the \big right “double-decker” parenthesis:

(CHARACTER H 11
(CHARWD R 0.4333284)
(CHARHT R 0.039999)
(CHARDP R 1.160013)
(NEXTLARGER H 13)
(MAP
(SELECTFONT D 1)
(MOVERIGHT R 0.0962952)(MOVEDOWN R 0.410001)
(PUSH)(SETCHAR O 51)(POP)
(MOVEDOWN R 0.600006)
(SETCHAR O 51)
)
)


This character is built stacking two right parentheses taken from cmr6. The initial idea was to make this character as high and deep as the \big parentheses of cmex10, and as wide as the parentheses of cmr6. The latter width is 0.481476 times 6.0 points (see the definition of character octal 51 in cmr6.pl; you can generate this file by typing

tftopl cmr6.tfm cmr6.pl


at the command line), that is, 2.888856 points in absolute dimensions, assuming that our font is loaded at its design size of 10.0 points and that, therefore, the cmr6 font is referenced at its design size of 6.0 points. Thus, we should indicate a relative width of 0.2888856. However, this width looks too narrow in print, so I decided to make the character box 50% wider, and also to allow for some clearance at the right side (the side that points toward the “contents” of the d.-d. parenthesis).

The typesetting instructions contained in the (MAP …) list take into account this clearance, as well as the fact that the height of the (right) parenthesis in cmr6 is 0.75 relative units (again, see cmr6.pl, character octal 51): after selecting font number 1, we start by moving right and down from the reference point by appropriate amounts (in calculating the vertical offset, don’t forget to convert between the different units used in the two fonts, that is, 10.0 points and 6.0 points), and then we proceed as above.

Similar adjustments are made for the two \Big characters, 12 and 13 (hexadecimal).

The characters in our virtual font are organized in two series of similar glyphs of increasing size: for the left d.-d. parenthesis, the series starts at character position 10 and continues through slots 12, 14, 16, 18, 1A, 1C, and 30 (numbers are in hexadecimal notation), and the other series, for the right parenthesis, runs in a similar way from slot 11 to slot 31. The last characters of both series (30 and 31) are extensible: for instance, character 30 is specified as

(CHARACTER H 30
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 1.760019)
(VARCHAR
(TOP H 30)
(MID H 38)
(BOT H 40)
(REP H 42)
)
)


The (VARCHAR …) list indicates, precisely, that character 30 (hexadecimal) is an extensible character made up of four pieces: a top piece found in slot 30 itself, a middle piece found in slot 38, a bottom piece found in slot 40 and a repeatable piece found in slot 42 (all hexadecimal numbers). All pieces but the middle one are mapped to the character with the same position in font 0, that is, cmex10 (this is the default mapping, so no (MAP …) list needs to be included in any of them), with identical width, height, and depth; their positions have been chosen so that each piece is mapped to the correct part of the extensible left parenthesis. The middle piece, instead, is defined as follows:

(CHARACTER H 38
(CHARWD R 0.875003)
(CHARHT R 0.039999)
(CHARDP R 3.560037)
(MAP
(PUSH)(SETCHAR H 40)(POP)
(MOVEDOWN R 1.800018)
(SETCHAR H 30)
)
)


This means a character constructed stacking the bottom piece of the same extensible paranthesis over its top piece, as wide and as high as the bottom piece, and with depth calculated in such a way to make the height-plus-depth of this character equal to the sum of the height-plus-depth of the bottom piece and of the height-plus-depth of the top piece (that is, to 1.800018 + 1.800018 = 3.600036). The (MAP …) list specifies exactly this construction, using the same technique described above.

The other extensible character (hexadecimal 31) is similar.

If you feel really enthusiastic about this solution, you can now repeat the whole process three more times to derive, from the cmex9, cmex8, and cmex7 fonts provided by the AMS (plus some suitable sizes, perhaps scaled, of the cmr font), three other virtual fonts similar to our ddeckparex10, but with a design size of 9, 8, and 7 points respectively. These additional virtual font should be named, of course, ddeckparex9, ddeckparex8, and ddeckparex7. After this, you could adapt the definition of the ddkparex font family contained in the ddkparex package so that it loads these new (virtual) fonts for certain size ranges, instead of scaling down the ddeckparex10 font. It suffices to mimick what the amsfonts package does for the cmex font: just substitute the single line

\DeclareFontShape{U}{ddkparex}{m}{n}{<->ddeckparex10}{}


with

\DeclareFontShape{U}{ddkparex}{m}{n}{
<-7.5>ddeckparex7
<7.5-8.5>ddeckparex8
<8.5-9.5>ddeckparex9
<9.5->   ddeckparex10
}{}


in the file ddkparex.sty.

This has been an exceedingly long answer! (I said it was going to be “pedantic”.) But this question actually demanded an answer involving virtual fonts: I would say this is exactly the kind of problem one would invent for the purpose of illustrating what virtual fonts could typically be used for.

I found that what I originally wanted to do would imply fiddling with .tfm files and the like. But thanks to another texSE question (How can I create a new extensible symbol?) I came up with this:

\documentclass{scrartcl}
\usepackage{scalerel}% http://ctan.org/pkg/scalerel
\newcommand*\doubledecker[2]{
\scaleleftright{\leftdoubleparens}{{\mathstrut #1 \atop \mathstrut #2}}{\rightdoubleparens}
}
\newcommand\leftdoubleparens{\vbox{\hbox{(}\vskip-2.85pt\hbox{(}}}
\newcommand\rightdoubleparens{\vbox{\hbox{)}\vskip-2.85pt\hbox{)}}}

\newcommand\test[2]{
$\frac{#1}{#2} {#1\choose #2} \doubledecker{#1}{#2}$
}
\begin{document}
\test{a}{b}
\test{pm}{bq}
\test{e^{2\pi i}}{x}
\test{e^{2\pi i}}{\sum_{n=1}^\infty\frac1{n^2}}
\test{e^{2\pi i}}{\displaystyle\sum_{n=1}^\infty\frac1{n^2}}
\end{document}


\else\ifx\scriptstyle#3\scriptscriptstyle
\fi\fi
\else\ifx\scriptstyle#3\scriptscriptstyle
\fi\fi
\ifx\displaystyle#3\expandafter\bigl\fi(
\hfil##\hfil
}}}}}
\makeatother
\usepackage{amsmath}% FOR COMPARISON
\begin{document}
$\binom{n}{m} \doubledecker{n}{m}\quad \binom{\sum x^2}{y_{y_y}} \doubledecker{\sum x^2}{y_{y_y}}$
$\scriptstyle\binom{n}{m} \doubledecker{n}{m}\quad \binom{\sum x^2}{y_y} \doubledecker{\sum x^2}{y_y}$
$\scriptscriptstyle\binom{n}{m} \doubledecker{n}{m}\quad \binom{\sum x^2}{y_y} \doubledecker{\sum x^2}{y_y}$
\end{document}