# vspace mysteriously being inserted after nested custom environment

I have created my own environment that creates a table using tabu and then puts some vertical space after it, unless my environment is being nested, in which case I want it to skip the vertical space. However, the vertical space seems to be getting inserted even after a nested use of my environment. What am I doing wrong?

MWE (though I wish it was more M):

\documentclass{article}

\usepackage{tabu}
\usepackage{environ}
\usepackage{xparse}

\makeatletter

\newcommand{\lxtabu@table}[1]{
\begin{tabu}{X}
#1
\end{tabu}%
}

\newcounter{lxtabu@counter}

\NewDocumentEnvironment {lxtabu} {}
{%
\stepcounter{lxtabu@counter}
\Collect@Body\lxtabu@table
}{%
\typeout{ending lxtabu}%
% Only output vspace if we're not in a nested lxtabu.
\ifnum\value{lxtabu@counter} = 0 %
% If you comment this out the extra vertical space between lines 2
% and 3 goes away.  (But so does the space between lines 3 and 4,
\typeout{inserting space}\vspace{\baselineskip}%
\fi
}

\makeatother

\begin{document}

\begin{lxtabu}
Line 1 \\
\begin{lxtabu}
Line 2 \\
\end{lxtabu}
\\
Line 3, should not have big empty space above it \\
\end{lxtabu}

Line 4, should have \verb|\baselineskip| space above it.

\end{document}


Result:

My desire is that the spacing between lines 2 and 3 should be the same as between lines 1 and 2.

I'm not positive that the \vspace is being "executed" when it shouldn't be, but if I change it to \vspace{0pt} the unwanted space between lines 2 and 3 goes away. The \typeout commands I added produce the expected result, with "inserting space" only appearing once, at the end.

A few more notes:

• I'm using LuaLaTex from MacTeX 2014, which is presumably built upon TeX Live 2014.

• I'm using \Collect@Body because \begin{tabu} and \end{tabu} give errors when used inside the beginning and end, respectively, of \NewDocumentEnvironment. Same problem with just \tabu and \endtabu.

• Though not shown in the MWE, I need \NewDocumentEnvironment for its argument parsing. However, I'd accept a solution that doesn't involve \NewDocumentEnvironment if I can still declare an environment that can parse optional arguments after mandatory ones, i.e. \begin{lxtabu}{Foo}[Bar].

• My lxtabu environment needs to make sure there is space between its bottom and whatever comes next, which could be text, another lxtabu environment, or perhaps something more exotic. I don't want to have to move the \vspace outside of my lxtabu environment.

• haven't tried this, but there are some infelicities in the definition that you should fix: insert a % after the opening brace in the "before" segment; put a space after the 0 in the check on \value{lxtabu@counter}; % isn't needed after \fi. i'd also suspect the double backslash after line 3 as participating in the addition of the blank line. a \par at the beginning of the "after" segment might be a better approach. Aug 7, 2014 at 18:05
• Thanks for the feedback. I've gone ahead and incorporated your suggested changes into the MWE. If I comment out the \\  at the end of line 3 it doesn't seem to make any difference to any of the spacing. Aug 7, 2014 at 18:18
• i've experimented some, adding diagnostic \message{...}%s in significant places, commenting out all double backslashes and blank lines, placing a % after every {ltxtabu}, and even putting \unskip at the end of line 3 to guard against the chance that a space at the end of that line makes it "too long". the "inserting space" is indeed coming only at the desired location, and there is no space before line 3, but, curiously, the vertical space before line 4 is doubled! i will try again later. the enlarged space before line 4 is a mystery to me. Aug 7, 2014 at 18:58
• regarding your comment on the answer that it would be "inconvenient for your users" to omit the double backslash on the last row before \end{ltxtabu}, there is a precedent -- in the multi-line math display structures provided by amsmath. (of course, if your users aren't setting math, that doesn't mean much.) Aug 7, 2014 at 19:01

The mystery is readily solved: tabu processes its input twice or more. It disables some commands during the trial processing, but it forgets to do it for \stepcounter and \addtocounter. One of the commands it disables is \write, with the consequence that the \vspace is added but the message is not issued.

\documentclass{article}

\usepackage{tabu}
\usepackage{environ}
\usepackage{xparse}
\usepackage{xpatch}
\usepackage{showframe}

\makeatletter

\patchcmd{\tabu@setstrategy}
{\let\color}
{}{}

\newcommand{\lxtabu@table}[1]{%
\begin{tabu}{|X|}
#1
\end{tabu}%
}

\newcounter{lxtabu@counter}

\NewDocumentEnvironment {lxtabu} {}
{%
\stepcounter{lxtabu@counter}%
\Collect@Body\lxtabu@table
}{%
\typeout{ending lxtabu}%
% Only output vspace if we're not in a nested lxtabu.
\ifnum\value{lxtabu@counter} = 0 %
% If you comment this out the extra vertical space between lines 2
% and 3 goes away.  (But so does the space between lines 3 and 4,
\typeout{inserting space}\vspace{1\baselineskip}%
\fi
\ignorespacesafterend
}

\makeatother

\begin{document}

\begin{lxtabu}
Line 1 \\
\begin{lxtabu}
Line 2 \\
\end{lxtabu}
\\
Line 3, should not have big empty space above it \\
\end{lxtabu}

Line 4, should have \verb|\baselineskip| space above it.

\end{document}


I don't know whether this patch can have other consequences; I never use tabu because it has several problems and its author announced that the next version will be largely incompatible with the present one.

I added the showframe package just to show the page frames and also vertical rules to tabu for making clearer what environment is concerned. The \ignorespacesafterend is better, although possibly not needed.

• Thank you, once again! I aspire to your level of skill at all things TeX. I am disturbed by tabu's apparently unmaintained status, and concerned that the touted new version of tabu may never surface. If you know of suitable replacement(s) for tabu, particularly an environment which does not require you to specify the table width up front (as tabularx), nor the column widths, I think I would be interested in weaning myself off tabu. Aug 7, 2014 at 20:57
• @dsedivec My opinion is that a table should be as wide as necessary. I have no real idea what your code is about, so I can't tell more. Aug 7, 2014 at 21:00

I get the desired result by removing some of your '\\', specifically the ones after line 2 and the one on the line before line 3:

\begin{document}

\begin{lxtabu}
Line 1\\
\begin{lxtabu}
Line 2 %\\
\end{lxtabu}
%\\
Line 3, should not have big empty space above it \\
\end{lxtabu}

Line 4, should have \verb|\baselineskip| space above it.

\end{document}


• Hm, that's interesting. Unfortunately I don't really like this as a "solution," as I think it would be inconvenient and error-prone for my users to have to remember to add a \\  at the end of every table row except for the last row. It is customary to terminate every row in a table with \\ , including the last row, right? Aug 7, 2014 at 18:06
• After looking at this further, taking out the \\  at the end of line 2 doesn't seem to have any effect. Taking out the \\  just before line 3 does have the effect you've displayed, but unfortunately that won't work for me as I need line 3 to be on its own row. (The real table is more complex.) However, I will note that when you comment out the \\  just before line 3 it does eliminate the space between lines 2 and 3, but it seems to increase the space between lines 3 and 4. Weird! Aug 7, 2014 at 18:11
• I would guess that removing the one before line 3 should be ok, since it will be on its own row by default since you are ending the environment with '\\\'. But it is hard to do check this without more code. Aug 7, 2014 at 18:17