L7x font encoding ellipsis issue

Question

Can someone explain why does compiler throw an error

\f@encoding=macro: ->T1. l.5 \makeatletter\show\f@encoding \makeatother

! TeX capacity exceeded, sorry [input stack size=5000]. \textellipsis ->\L7x-cmd \textellipsis \L7x\textellipsis l.6 \dots

If you really absolutely need more capacity, you can ask a wizard to enlarge me.

for this?

\documentclass{article}
\usepackage[L7x,T1]{fontenc}
\usepackage[utf8]{inputenc}
\begin{document}
\makeatletter\show\f@encoding\makeatother
\dots
\fontencoding{L7x}\selectfont
\makeatletter\show\f@encoding\makeatother
\dots
\end{document}

It seems that for the first \dots it goes into L7x table while the current font encoding is T1.

Is that an expected behavior or a bug?

Kind regards, Ivan

Fix Proposal

Based on the analysis @David Carlisle has provided below, after digging into other encoding definitions and fntguide I would like to present my proposal for fixing the issue within l7xenc.def. WYSIWYM could be easily achieved replacing all these

\bgroup
  \def\default@family{lmr}%
  \DeclareFontEncoding{L7x}{}{}
\egroup
\@ifundefined{L7x@default@family}{\@namedef{L7x@default@family}{lmr}}{}
\def\hooked@encoding{L7x}
\DeclareRobustCommand\selectfont
  {%
  \ifx\f@linespread\baselinestretch 
  \else
    \set@fontsize\baselinestretch\f@size\f@baselineskip 
  \fi
  \ifx\f@encoding\hooked@encoding
    \expandafter\ifx\csname \f@encoding+\f@family\endcsname\relax
      \edef\reserved@a{\lowercase{\noexpand\IfFileExists{\f@encoding\[email protected]}}}%
      \reserved@a\relax{%
        \expandafter\ifx\csname\f@encoding @default@family\endcsname\relax
    \else
          \@font@info{Failed loading font information for \f@encoding+\f@family.}%
      \expandafter\edef\expandafter\f@family\expandafter{\csname\f@encoding @default@family\endcsname}%
    \fi
    }%
    \fi
  \fi
  \xdef\font@name{\csname\curr@fontshape/\f@size\endcsname}%
  \pickup@font
  \font@name
  \size@update
  \enc@update
  }

by only these three lines:

\DeclareFontEncoding{L7x}{}{}
\DeclareErrorFont{L7x}{lmr}{m}{n}{10}
\DeclareFontSubstitution{L7x}{lmr}{m}{n}

In order not to get warnings (saying cmr is substituted by lmr) one should explicitly use statement \usepackage{lmodern} before calling for fontenc, satisfying what's written in fntguide:

The font specification for <encoding><family><series><shape> must have been defined by \DeclareFontShape before the \begin{document} is reached.

Closure

I'm glad to inform you all that the above Fix Proposal has been included in the latest version of lithuanian bundle (version 2017-12-06) maintained by Sig­i­tas Tolušis, along with making babel's lithuanian agnostic of font and font encoding. Additionally, missing translations for Proof and Glossary have been defined.

Answer 1

At the top of the file l7xenc.def you will find the lines

\bgroup
  \def\default@family{lmr}%
  \DeclareFontEncoding{L7x}{}{}
\egroup

defining \default@family in a group doesn't do much and defining the L7x encoding inside a group does not work at all, effectively the encoding is not defined (but enough bits of it get defined that you avoid the high level error checks).

If you change that to

%\bgroup
%  \def\default@family{lmr}%
  \DeclareFontEncoding{L7x}{}{}
%\egroup

then your example works, but this should be raised with the maintainers of the file, I don't know the intent of the original code.

Specifically what happens is that any encoding specific commands are optimised for the usual case that any given use is usually the same encoding as the previous use, however they are guarded by an internal command \<enc>-cmd that detects if a different encoding is in force and switches the command definition.

so \textellipsis has the definition

\L7x-cmd \textellipsis \L7x\textellipsis

which says (or at least is supposed to say) if the encoding is L7x use the optimised version for that encoding, otherwise find a version for the current encoding.

Unfortunately the definition of \L7x-cmd was lost at the \egroup the line after it was defined, this leaves it defined as \relax which does nothing

so \textellipsis does \relax then calls \textellipsis again and you recurse until the stack fills.