Inconsistent behavior of boxes in amsmath's gather.

Recently I was working on a package (ytableau) and had a strange experience with the gather environment in amsmath. Namely, it screws with \vtop and, seemingly, with boxes that I reserve before entering the environment. Consider the following minimal test cases:

\documentclass{article}
\usepackage{amsmath}
\begin{document}
\newbox\mybox
\setbox\mybox=\vtop{\halign{&#\cr a & b \cr c \cr}}
ht = \the\ht\mybox, dp = \the\dp\mybox
\begin{gather}
ht = \the\ht\mybox, dp = \the\dp\mybox
\box\mybox
\end{gather}
\begin{gather}
\setbox\mybox=\vtop{\halign{&#\cr a & b \cr c \cr}}
ht = \the\ht\mybox, dp = \the\dp\mybox
\box\mybox
\end{gather}
\end{document}


The result should be as follows: you will see printed "ht = 6.94444pt, dp = 12.0pt" before the first gather, but inside it you will see only ht = 0.0pt, dp = 0.0pt. Where did my box go? If you insert the same diagnostic after gather you will note that it does not come back.

In the second gather, you will see "ht = 0.0pt, dp = 21.94444pt" and the \halign will print an "a b" over a "c" as desired, but the whole thing will indeed be aligned so that the very top of the first line is at the very bottom of the text line established by the environment (compare with the other text and the label). Where did the height go? (Don't say "into the depth" :). )

This is really annoying because it absolutely violates what I understand \vtop to do: put the baseline of the box at the baseline of the top interior box. Printing that same \halign outside of gather (in a \vtop) puts the "a b" in the text, as it should be.

This bug does not arise inside align. What did happen there (and in gather) was that I found that some of my macros were being double-evaluated. Presumably the AMS environments read their contents twice to get the proper typesetting? Unfortunately, I can't conceive of how that would cause the second error (note that the numbers change even if you don't print the box).

-

The gather environment does indeed seem to evaluate its environment twice. It is the second evaulation that actually results in the final typeset result. Replace \box\mybox by \copy\mybox to work around the problem.

Edit: I neglected your second problem. It looks like the gather environment sets \everycr for its own purposes, and that interferes with your ´\halign. One fix is to use

\setbox\mybox=\vtop{\everycr{}…}


or to move the whole assignment outside the gather env.

Edit the 2nd: Remembering that the inside of a \vtop is also a group, I simplified the solution by moving \everycr inside it.

-
That does seem to solve the first one, but not the second. For your amusement: replace \box\mybox in the second gather simply by \vtop{\halign{&#\cr a & b \cr c \cr}} and observe the same problem. This one is not an issue with box registers, I think. It IS a problem with \halign, though, as the problem does NOT occur when you write simply \vtop{\hbox{a b}\hbox{c}}. I am so confused. –  Ryan Reich Nov 9 '10 at 13:00
Fantastic! That works just fine. Note: it is not possible to move the assignment outside the environment because it's part of a large construction in a macro that the user should expect to be able to call wherever they like. But the \everycr thing seems to do the trick. –  Ryan Reich Nov 9 '10 at 13:07
@Ryan: Please note my second edit. It's simpler than my first solution. –  Harald Hanche-Olsen Nov 9 '10 at 13:43
That's nice, it also simplifies something else I had to do in the package. I think. –  Ryan Reich Nov 9 '10 at 13:52

the gather environment definitely evaluates its content twice. from the amsmath.dtx file:

All display environments get typeset twice---once during a measuring'' phase and then again during a production'' phase; \cs{ifmeasuring@} will be used to determine which case we're in, so we can take appropriate action.

if there are bugs in any of the amsmath environments -- and indeed this does have all the marks of a bug (as confirmed by the first answer) -- please bring them to the attention of tech-support@ams.org. we have an extensive list that is waiting for an update to be scheduled, and i have added this report to the list. i have also recorded this location so that any suggestions posted here will become part of the record that can be reviewed by the developer when work actually begins.

-
sorry to sort of hijack the thread, but can you take a look at tex.stackexchange.com/questions/3799/… ? It is about some undocumented behaviour of amsthm`, so I am not sure if it is a bug or not. Oh, and thanks for pointing out that e-mail address, I hadn't known that there's a way to file bugs with the AMS. –  Willie Wong Nov 10 '10 at 0:08
@willie: i've looked at the question you cite, and will post some comments there; the situation is rather more complicated than noted, and yes, the documentation is weak (improvement is contemplated). re the address for reporting bugs to ams, it's given at the top of every package file we distribute -- look there first. –  barbara beeton Jan 6 '11 at 17:18