22

I seem to often run into capacity issues and have been upping the limits each time. There are several question on this site pertaining to these. However, there seems to be some conflicting information and no single place where all these limits are detailed.

Main Question:

So, what I am looking for is

  1. Name of the variable
  2. Default value.
  3. Absolute max that it can be set to.
  4. What steps can I take to minimize these usages. What are the DOs and DONTs for each these capacity limit.

The settings I am aware of are the ones reported at the end of the log file:

 106477 strings out of 493042
 2524840 string characters out of 39884661
 12363848 words of memory out of 12435455
 103471 multiletter control sequences out of 15000+600000
 197845 words of font info for 228 fonts, out of 8000000 for 9000
 1143 hyphenation exceptions out of 8191
 86i,17n,86p,10523b,3398s stack positions out of 5000i,500n,10000p,200000b,80000s

but there may be others that are not listed here. My immediate issue is with words of memory as the numbers above are from the first run of my indexing. If I attempt to do a second run then I get TeX Capacity Exceeded.

Examples of Other Variables:

From Davis Carlisle's comments, there is also extra_mem_top and extra_mem_bot that can also be used. But, does this allow for larger words of memory, or are these related to one of other capacity limits in the report. Again, what is the maximum I can set these to?


Possible DOs and DONTs:

Some of the questions I have in regards to minimizing the usage are listed here, so perhaps these can be covered in the DOs and DONTs for each of the variables.

  1. Is it better to use \def\foo{xxx} instead of \newcommand*{\foo}{xxx}. I can perform the check for duplicate macro names outside of TeX.

  2. Do macro names with a large number of characters increase usage of some of these?

    For instance, does \expandafter\newcommand\csname dir1/dir1/long-file-name\endcsname{xxxx} have any issues in regrads to capacity. Would it be better for me to map dir1/dir1/long-file-name to some shorter string such as ZXyw to save on these limits. Yes, readability is lost, but I can automate this replacement.

  3. Does repeated text result in increased use of the limits: Which of the following would be better:

    \MyMacro{../../../../dirA/dirB/dirC/dirD/file0001.tex}
    \MyMacro{../../../../dirA/dirB/dirC/dirD/file0002.tex}
     ...
    \MyMacro{../../../../dirA/dirB/dirC/dirD/file5000.tex}
    

    or

    \def\Path{../../../../dirA/dirB/dirC/dirD/}
    \MyMacro{\Path/file0001.tex}
    \MyMacro{\Path/file0002.tex}
     ...
    \MyMacro{\Path/file5000.tex}
    
  4. Does \undefing macros after they are no longer needed help with these limits?

Certainly, I can experiment with these things that I have thought of, but perhaps there are other things that can be done to reduce the useage of these limited string and words.


Example of Inconsistency


Notes:

  • I am assuming that any changes would be made to the file reported by kpsewhich texmf.cnf as per Leo Liu's answer at How to expand TeX's “main memory size”? (pgfplots memory overload), which in my case is

    /usr/local/texlive/2015/texmf.cnf
    

    If there are different ways to set some of these I would like to know those as well.

  • My main problem right now is with indexing a large number of entries and the words of memory limit. I am less concerned about run time (within reason), more about exceeding these limits.
  • I am using TeXLive2015 in case it matters. But if the limits are dependent on the distribution it would be good to know that as well.
  • This question is not intended to be about error conditions that result in exceeding the capacity limits (such as my naive question Tex Capacity Exceeded (if remove % after use of macro)).

References:

  • \expandafter\newcommand*\csname… will expand * rather than \csname. – Henri Menke Feb 11 '16 at 9:43
  • @HenriMenke: opps, have corrected that. – Peter Grill Feb 11 '16 at 9:45
  • I think the 8000000 refers to main_memory but you can up extra_mem_top and extra_mem_bot so the effective memory is beyond that. (or use luatex which uses dynamic allocation) – David Carlisle Feb 11 '16 at 11:08
  • @DavidCarlisle: But the question is what is extra_mem_top and extra_mem_bot? It isn't obvious to me. What is the difference between these extra memory versus the main_memory. What are the maximum values I can set these two. What will increasing these allow me to do? – Peter Grill Feb 11 '16 at 11:55
  • @PeterGrill I think it's just a way of not changing too much of the original pascal tex.web memory layout. Externally in macro usage it's no difference but internally it's an escape hook to allow more "main" memory without having to change the memory layout of everything else to fit it in. Increasing them allows you to have more main memory (beyond the 8000000) – David Carlisle Feb 11 '16 at 12:07
6

Is it better to use \def\foo{xxx} instead of \newcommand*{\foo}{xxx}. I can perform the check for duplicate macro names outside of TeX.

No, comes to the same thing

For instance, does \expandafter\newcommand\csname dir1/dir1/long-file-name\endcsname{xxxx} have any issues in regards to capacity.

in theory yes, the csname has to be stored somewhere so this eats in to 2524840 string characters out of 39884661

but you are presumably only going to make it 20 or so characters less so not much difference out of 39884661

Which of the following would be better:

again marginally the first uses more as it defines one more csname (\path) which takes one out of

 103471 multiletter control sequences out of 15000+600000

but you are nowhere near that limit

Does \undefing macros after they are no longer needed help with these limits?

yes that's why latex removes lots of commands at \begin{document} the hash entry is not reclaimed (number of multi-letter commands) but the main memory used to save the definition is reclaimed.

  • That is useful info, but does't address the main thrust of the question of what are the absolute maximums and what programming practices to follow to keep their usage down. Also, my particular use case makes tens of thousands of calls to` \index` so saving 20 here and there helps. Am experimenting now, and replacing the long paths in with macros reduces the words of memory from 12,400,429 to 11,458,517. – Peter Grill Feb 11 '16 at 11:16
  • @PeterGrill also I realise i misread long macro names for \long macro names So I'll delete one "no" and just leave the following quote. Not sure what to say about the actual limits they are all in texmf.cnf – David Carlisle Feb 11 '16 at 11:23
  • @PeterGrill any reduction in main mem from using a macro in the argument to MyMacro is related to whatever that macro is doing not to the call shown. if you for example you save that argument in another definition, that definition will have a smaller replacement text so use less memory. If you just use the argument for typesetting or writing to a file, the fact that the argument was passed as a macro should make no difference. Indexing is normally just a matter of writing to a file and shouldn't use that much memory at all? – David Carlisle Feb 11 '16 at 11:29
  • The above results are just from indexing -- nothing else. My indexing has a lot of formatting applied so that is probably what is causing the problem. And the reduction almost 1 million words of memory is strictly from replacing repeated text with a macro -- exactly as shown in the question. No other there differences in terms of what the index looks like. So, my results seem to be contradicting your last comment -- perhaps I am not understanding the last comment. – Peter Grill Feb 11 '16 at 11:49
  • Also, I don't see where in the absolute MAX settings of these variables are defined in the texmf.cnf. – Peter Grill Feb 11 '16 at 11:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.