Tell me more ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.
up vote 6 down vote favorite

I'm trying to read a large file (about 1.5M) with information to be processed, but it always ends prematurely with either \input or a loop with \read. The file is correctly balanced (believe me) and by removing parts of it the file ends at different points, even in the middle of a macro name (with the corresponding error of undefined macro), which, of course, is complete in the source. So the problem is TeX itself (TeX Live 2010). I presume there should be a parameter setting a limit, but after reading the documentation I could find nothing. The program has nothing special except a lot of \pdfsavepos. Is there a parameter in TeX Live to change the file size limit or is there a known bug?

More on the error

The code is:

\begingroup
\makeatletter
\@tempswatrue
\loop
\ifeof\ulp@in\@tempswafalse\fi
\if@tempswa
  \read\ulp@in to\@tempa
  \@tempa
\repeat
\endgroup

When the following part is read:

\ulp@Ga{24}\@ne {%
\ulp@Pa{1}{13349836}{12624364}{0}%
\ulp@Pa{2}{14554971}{12624364}{1}%
\ulp@Pa{3}{14739040}{12624364}{1}%
\ulp@Pa{4}{16705126}{12624364}{1}%
\ulp@Pa{5}{16889195}{12624364}{1}%
\ulp@Pa{6}{18218124}{12624364}{1}%
\ulp@Pa{7}{7030046}{11837932}{1}%
\ulp@Pa{8}{8943337}{11837932}{0}%
}%

the file is prematurely ended, as shown in the log file:

\@tempa ->\ulp@Ga {24}\@ne {\ulp@Pa {1}{13349836}{12624364}{0}\ulp@Pa {2}{14554971}{12624364}{1}\ulp@Pa {3}{14739040}{12624364}{1}\ulp@Pa {4}{16705126}{12624364}{1}\ulp@Pa {5}{16889195}{64}{1}\ulp@Pa {5}{16889195}{

Note \read has not read correctly the file, and the last argument of \ulp@Ga is incomplete (the last thing is {). Before this point, all \ulp@Gas are read correctly (some of them are even longer).

share|improve this question
1  
Can you see at which point exactly it ends? Always at the same offset from the start? – Paŭlo Ebermann Aug 2 '11 at 14:42
2  
This sounds strange. TeX typesets the first part of an input file before it even reads most of the rest of it. It should be able to process files of any lengths. Of course if you wrap half the file content in a macro argument in must be read completely before its content is processed. – Martin Scharrer♦ Aug 2 '11 at 15:34
@Paŭlo After investigating a little further it seems it's at the same offset from the end. Much like if the last chunk were discarded in a buffered reading. In fact, the error happens even with small files, larger than (more or less) 8K (below that, it's ok, and I'm testing only \read). – Javier Bezos Aug 2 '11 at 15:42
Post your reading code - maybe there is a problem with this. – Paŭlo Ebermann Aug 2 '11 at 15:45
@Martin I'm puzzled and lost. I've copied above a few lines from the processed file. – Javier Bezos Aug 2 '11 at 15:46

1 Answer

up vote 8 down vote

At last! After struggling three days with this problem I've realized where the problem was -- the file to be read was not still really closed when it was read. The asyncronous nature of TeX, you know. I just forgot \closeout doesn't actually closes the file and waits until a page is shipped out (for example with an explicit \clearpage). This meant the last chunk of the file was still buffered and not written. Thank you very much for you interest.

share|improve this answer
2  
In this case you should use \immediate\closout\.... – Martin Scharrer♦ Aug 2 '11 at 16:57
There is \immediate which you can prepend to \openout, \write and \closeout. (But then you can't use any page-dependent stuff in the output, of course.) – Paŭlo Ebermann Aug 2 '11 at 16:58
Of course. Instead I'll close the file in \AtEndDocument and move the processing of the file to the aux file (with an in-between \clearpage). That is \AtEndDocument{\closeout\ulp@out\write\@auxout{\string\ulp@enddoc}} (where \ulp@endoc processes the file just closed). Once I've found where the problem was, it's easy to fix it. – Javier Bezos Aug 2 '11 at 17:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.