14

I have a template with an extensive documentation. (See TemplateDocumentation.pdf). It compiled several month ago. Now with updated packages it does not compile anymore because of the famous No room for a new \write problem.

Before everything was already tuned such that all writes are used. I had to disable packages to make it compile (and thus give up functionality). Now I can test that disabling for example pgf/tikz solves the original problem. But I would rather not give up pgf/tikz in my template... Note that I do not use the external package of tikz.

So the basic question is how to find out which packages are actually using \write. Some must have introduced an file output or consume more than before.

The code fails at that first time the filecontents package is used.

A minimal example for this problem is currently not available and would be really hard to create, Basically because the code basis is so huge.

I use this code to make writes less a problem, but it solves nothing really

\usepackage{morewrites}
% see http://www.tex.ac.uk/cgi-bin/texfaq2html?label=noroom
\usepackage{etex} 
\reserveinserts{28}

The packages I know of which use a \write and which I load are

  • tikz/pgf
  • imakeidx
  • glossaries
  • listings
  • titletoc

The complete list of packages can be found in my documentation (see link) on page 240.

  • 2
    etex can't help, the maximum number of output streams is still 16. – egreg Jan 18 '14 at 22:02
  • 6
    As from v4.0 glossaries uses one extra write if you define entries in the document instead of the preamble. See the section Drawbacks With Defining Entries in the Document Environment in the glossaries user guide for further details. – Nicola Talbot Jan 18 '14 at 22:07
  • 2
    As a fan of »KOMA-Script« you might be interested in its »scrwfile« package. It is described in the user guide. – Thorsten Donig Jan 19 '14 at 8:03
  • @egreg: Do I understand correct that loading the etec package has no effect? I added these lines because they are recommended by the FAQ. – Matthias Pospiech Jan 20 '14 at 18:46
  • Loading etex has no effect for the number of input and output streams that can be allocated and simultaneously open. This number is still 16 in all TeX engines (but, of course, LuaTeX can use different ways for writing to files). With etex you can access 32768 registers of the various types (counters, dimensions, boxex and so on) instead of the traditional 256. – egreg Jan 20 '14 at 18:54
18

Without a minimal working example (MWE) this is just guessing, but if your document has defined acronyms or glossary entries within the document environment and you upgraded to version 4.0, an extra write is required. The reason for this is as follows:

Version 3.07¹ and below wrote the name, description and symbol fields to the external glossary files (in addition to the sort field). There are multiple issues with this:

  1. Fragile commands within those fields need protecting otherwise an error occurs when the command is written to the file. To overcome this, glossaries had the sanitize package option which basically converted the contents of those fields to a list of letters, so for example, if you had \cite in the description field this would be converted from a macro into the five characters \ c i t and e. This can now be safely written to the glossary file but if you attempted to display the description elsewhere in the document you'd actually get "\cite" in your PDF (or, more usually, "—cite" since the default serif font displays the character \ as an em-dash).

  2. Any occurrences of makeindex or xindy special characters need to be escaped. This is done by glossaries but the longer the name, description, and symbol fields, the longer this takes to do, so it slows the document build.

  3. makeindex has a limited buffer and a long description can exceed this.

  4. A custom glossary style may be required that should display other fields, but these can't be accessed if the entry hasn't been defined.

To overcome these issues version 4.0 changed the way entry information was written to the glossary file. Now only the sort field and \glossentry{label} are written to the file. It's then up to the glossary style to define \glossentry so that it accesses the relevant information using commands like \glsentrydesc. This means there are no longer any worries about using fragile commands, since they don't get written to the file. (The sort value is automatically sanitized by default and the label can't contain fragile commands anyway.) Since indexing information is now in a shorter string it's a lot quicker to do any escaping of xindy/makeindex characters (in case they appear in, say, the sort field or entry label). And there's no need to worry about exceeding makeindex's buffer (unless you have an excessively long sort value or label).

There is, however, a drawback with this approach: all entries must be defined when the glossary is displayed in order for \glossentry to access the relevant fields. If the glossary is displayed at the end of the document, this isn't a problem, but quite often users like to have a list of acronyms or notation in the front matter. To get around this, and to ensure backward compatibility, glossaries modifies the definition of \newglossaryentry at the start of the document environment so the glossary information is written to a temporary file \jobname.glsdefs, which is then input at the start of the document on the next LaTeX run.

This ensures that the entry information is available when displaying the glossary, but there is the drawback of an extra write, expansion issues can occur and any changes to the definitions don't get picked up until the next run. The see key also doesn't work as it's not saved in a field (it simply triggers the cross-referencing when the entry is defined) and so doesn't get written to the .glsdefs file. It also means that terms defined using \newacronym are converted to a regular entry definition, which can cause a problem for the abbreviation mechanism.

See also Drawbacks With Defining Entries in the Document Environment in the glossaries user manual.

MWE1:

\documentclass{article}

\usepackage{glossaries}

\makeglossaries

\newglossaryentry{sample}{name={sample},description={an example}}

\begin{document}

\gls{sample}.

\printglossaries

\end{document}

Here glossaries defines two writes: one for the .ist file and one for the .glo file.

MWE2:

\documentclass{article}

\usepackage[acronym]{glossaries}

\makeglossaries

\newglossaryentry{sample}{name={sample},description={an example}}
\newacronym{xyz}{XYZ}{long form}

\begin{document}

\gls{sample}. \gls{xyz}.

\printglossaries

\end{document}

Here glossaries defines three writes: one for the .ist file, one for the glo file and one for the .acn file.

MWE3:

\documentclass{article}

\usepackage[acronym]{glossaries}

\makeglossaries

\begin{document}

\newglossaryentry{sample}{name={sample},description={an example}}
\newacronym{xyz}{XYZ}{long form}

\gls{sample}. \gls{xyz}.

\printglossaries

\end{document}

Here glossaries defines four writes: one for the .ist file, one for the .glo file, one for the .acn file and one for the .glsdefs file.

MWE4:

\documentclass{article}

\usepackage[acronym,index,symbols,numbers]{glossaries}

\makeglossaries

\begin{document}

\newacronym{xyz}{XYZ}{long form}

\gls{xyz}.

\printglossaries

\end{document}

Here glossaries defines seven writes: .ist, .glo, .acn, .idx (from the index option), .nlo (from the numbers option), .slo (from the symbols option) and .glsdefs (because \newacronym is in the document environment).

The glossaries-extra package provides the option docdef=restricted which allows document definitions but doesn't create the .glsdefs file. This means that the terms must be defined before the glossaries are displayed, but it means that an extra \write isn't required.

MWE5:

\documentclass{article}

\usepackage[acronym,docdef=restricted]{glossaries-extra}

\makeglossaries

\setabbreviationstyle[acronym]{long-short}

\begin{document}

\newglossaryentry{sample}{name={sample},description={an example}}
\newacronym{xyz}{XYZ}{long form}

\gls{sample}. \gls{xyz}.

\printglossaries

\end{document}

Now there are three writes: one for the .ist file, one for the .glo file and one for the .acn file.

How to Minimise the Number of new \writes

  1. Move all instances of \newglossaryentry (and \newacronym which uses \newglossaryentry) to the preamble or to a file that's loaded in the preamble either via \input or \loadglsentries (don't use \include).
  2. Make sure you haven't defined a glossary that you don't intend to use. For example:

    \documentclass{article}
    
    \usepackage[acronym]{glossaries}
    
    \makeglossaries
    
    \newacronym{xyz}{XYZ}{long form}
    
    \begin{document}
    
    \gls{xyz}.
    
    \printglossaries
    
    \end{document}
    

    Here glossaries defines three new writes: one for the .ist file, one for the .glo file and one for the .acn file, but one of these (.glo) isn't being used. This is a waste of a \write so suppress the main glossary using the nomain package option:

    \usepackage[nomain,acronym]{glossaries}
    
  3. If you don't need to make any further modifications to the .ist file, allow glossaries to generate it on the first build:

    \documentclass{article}
    
    \usepackage[nomain,acronym]{glossaries}
    
    \makeglossaries
    
    \newacronym{xyz}{XYZ}{long form}
    
    \begin{document}
    
    \gls{xyz}.
    
    \printglossaries
    
    \end{document}
    

    Then suppress it's creation by adding \noist to your document:

    \documentclass{article}
    
    \usepackage[nomain,acronym]{glossaries}
    
    \noist
    \makeglossaries
    
    \newacronym{xyz}{XYZ}{long form}
    
    \begin{document}
    
    \gls{xyz}.
    
    \printglossaries
    
    \end{document}
    

    Now glossaries only defines one new write for the .acn file.

  4. Use the savewrites package option. This uses just one write for the .ist and all the .glo, .acn etc files (but not for the .glsdefs file). With this option glossaries stores all the glossary information in tokens until the end of the document and then iterates through each glossary and writes to each of the glossary files reusing the same write register. This slows the document build, but provided you define all your entries in the preamble glossaries only defines the one write register.

    This method is susceptible to TeX's asynchronous output routine and can cause the locations to be incorrect.

    This option slows the build because whenever you reference an entry (using commands like \gls or \glsadd) information is appended to a token register. At the end of the document, glossaries then iterates through each defined glossary and writes the contents of the associated token register to the external glossary file. This takes longer than simply writing a line every time you use a command like \gls or \glsadd. How much longer it takes depends on the number of times you reference each entry.

As from glossaries version 4.04, there's another option that doesn't create any new files, as the sorting and indexing is done by TeX rather than using an external indexing application:

\documentclass{article}

\usepackage[acronym]{glossaries}

\makenoidxglossaries

\newglossaryentry{sample}{name={sample},description={an example}}
\newacronym{xyz}{XYZ}{long form}

\begin{document}

\gls{xyz}. \gls{sample}.

\printnoidxglossaries

\end{document}

This just creates the standard .log and .aux files (as well as the .pdf/.dvi file). In this case, \newglossaryentry and \newacronym are made preamble-only commands so there's no .glsdefs file. However, this method is very inefficient, has difficulties with hierarchical entries and can only sort according to the basic Latin set. It's therefore not recommended except as a last resort.

The glossaries-extra package provides two other methods that don't use any \write registers:

Just use \printunsrtglossary (or \printunsrtglossaries) to display the glossaries. This simply iterates over all defined entries (so they need to be defined first). There's no sorting and no location lists.

\documentclass{article}

\usepackage{glossaries-extra}

\newglossaryentry{zoo}{name={zoo},description={sample description}}
\newglossaryentry{aardvark}{name={aardvark},description={sample description}}

\begin{document}

\gls{aardvark}.

\printunsrtglossaries

\end{document}

There are no \write registers created here, but the glossary is in order of definition and includes both entries even though only one has been used in the document.

The other option is to use glossaries-extra with bib2gls. This involves a slight modification to the above. The entries are now defined in a .bib file. For example, entries.bib:

@entry{zoo,name={zoo},description={sample description}}
@entry{aardvark,name={aardvark},description={sample description}}

The document file now needs the record option and \GlsXtrLoadResources to specify the .bib file:

\documentclass{article}

\usepackage[record]{glossaries-extra}

\GlsXtrLoadResources[src=entries]% terms defined in entries.bib

\begin{document}

\gls{aardvark}.

\printunsrtglossaries

\end{document}

This doesn't define any \write registers (all the information is written to the .aux file) but now only the indexed entry (aardvark) appears in the glossary, the glossary is sorted and the entries have location lists. (Although in this case there's only one entry.)

The document build process (assuming the document file is called myDoc.tex) is:

pdflatex myDoc
bib2gls myDoc
pdflatex myDoc

(Replace pdflatex with xelatex etc, as appropriate.) If you want letter groups, you'll need the --group (or -g) switch:

bib2gls --group myDoc

This works in a different manner to the makeindex/xindy approach. bib2gls reads the required information from the .aux file and creates a file with the entry definitions that's input by \GlsXtrLoadResources. The entry's are first sorted before writing their definitions and only those indexed in the document are included. This means that the glossary can simply be displayed using \printunsrtglossaries, as in the previous example. bib2gls creates a transcript file (.glg) and one .glstex file per resource command (\GlsXtrLoadResources) but no \write register needs allocating for any glossary files.

There's a table comparing the various methods in the glossaries user manual.

By way of comparison, bib2gls comes with some example documents, including sample-multi2.tex which has 14 glossaries and 1 index. The main glossary is suppressed with nomain, the index glossary is created with the index package option and the remaining glossaries are created with \newglossary*:

\usepackage[record,% use bib2gls
 section,% use \section* for glossary headings
 postdot,% insert dot after descriptions in glossaries
 nomain,% don't create 'main' glossary
 index,% create 'index' glossary
 nostyles,% don't load default styles
% load and patch required style packages:
 stylemods={list,mcols,tree,bookindex}
]{glossaries-extra}

\newglossary*{bacteria}{Bacteria}
\newglossary*{markuplanguage}{Markup Languages}
\newglossary*{vegetable}{Vegetables}
\newglossary*{mineral}{Minerals}
\newglossary*{animal}{Animals}
\newglossary*{chemical}{Chemical Formula}
\newglossary*{baseunit}{SI Units}
\newglossary*{measurement}{Measurements}
\newglossary*{film}{Films}
\newglossary*{book}{Books}
\newglossary*{person}{People}
\newglossary*{mediacontrol}{Media Control Symbols}
\newglossary*{information}{Information Symbols}
\newglossary*{weather}{Weather Symbols}

The document loads hyperref, so this means that a write register (\@outlinefile) is created for hyperref's .out file in addition to the standard write registers \@mainaux and \@partaux defined by the LaTeX kernel. (If a table of contents is added, this would create a additional write register for the .toc file.)

If the document were changed to use makeindex/xindy, then it would require 1 write register for the style file and one per glossary (including the index) = 1 + 15 = 16. With the 2 registers defined by the kernel and hyperref's register for the PDF bookmarks, this means that the document would require 19 write registers (or 20 if the .glsdefs file is created if any glossary definitions are moved to the document environment).

With bib2gls only the 2 registers defined by the kernel and hyperref's .out register are created giving a total of 3 write registers, which is a significant saving.

In terms of associated temporary files, there are the standard .aux and .log files and hyperref's .out file. With makeindex/xindy there's also the style file and three files per glossary (input, output and transcript), which gives a total of 2 (.aux and .log) + 1 (.out) + 1 (.ist/.xdy) + 3 × 15 = 49.

With bib2gls, there's the transcript .glg file and 1 .glstex file per \GlsXtrLoadResources. The number of resource commands isn't related to the number of glossaries. One resource command may process multiple glossaries (if they have the same settings) or one glossary may require multiple resource commands (if it's divided into independently sorted blocks). In the case of sample-multi2.tex there are 9 resource commands, so the total number of associated temporary files = 2 (.aux and .log) + 1 (.out) + 1 (.glg) + 9 (.glstex) = 13. (The result sample-multi2.pdf is also available on CTAN.)

So using bib2gls not only removes the need for write registers associated with the glossaries but can also reduce the total number of temporary files.


¹ Versions between 3.07 and 4.0 were experimental and weren't uploaded to CTAN.

  • What is the slow down by using savewrites? The problem was now solved by moving all definitions in the examples before \begin{document}. This however required some more work on my latex examples package... Using nomain was no option since I create all possible lists in my documentation. – Matthias Pospiech Jan 19 '14 at 21:48
  • Just tested savewrites it fails compilation in your code at \RequirePackage{glossaries-compatible-307} with the error \glswritefiles undefined. – Matthias Pospiech Jan 19 '14 at 21:51
  • @MatthiasPospiech what version of glossaries are you using? – Nicola Talbot Jan 20 '14 at 14:14
  • @NicolaTalob Version 4.02 shipped by Miktex. – Matthias Pospiech Jan 20 '14 at 18:44
  • @MatthiasPospiech Should be fixed in 4.03. Allow a few days for the new version to propagate through the mirrors and get into the TeX distributions. – Nicola Talbot Jan 20 '14 at 20:19
4

The problem with morewrites and the filecontents package, is the way filecontents defines a \newwrite using \ch@ck7\reserved@c\write and by-passing the morewrites mechanism.

Here, is a hacked version which I use and by-passes the problem. Either Bruno or Scott Pakin need to fix it at the package level.

  • this is a lifesaver -- but why does your hacked package comment out the lines at the end that expose the change (without those lines it uses the default no-package filecontents) – Aubrey Blumsohn Jul 31 '15 at 9:42
  • @AubreyBlumsohn I was testing it with something else and commented the lines. Will fix, in case someone else finds this thread. – Yiannis Lazarides Jul 31 '15 at 12:34
  • @YiannisLazarides This was very helpful. I was just about to open another question "Is the morewrites package compatible with the filecontents package?" Now I know. – Jonathan Komar May 10 '16 at 6:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.