Read arbitrary lines from file

Question

Is there an easy way to read arbitrary lines from a file? For example "read line number 12" or "read backwards from last line to first line" or "read lines 1,3,5...".

Answer 1

No, you can't read arbitrary lines, but you can't do that in most other programming languages either. You have to read line after line from the beginning. Apart from the \read primitive, there is also the \readline primitive which reads a line verbatim. You can use that to save all lines in macros and use these afterward:

\newread\myread
\newcount\linecnt
\openin\myread=file.dat
\@whilesw\unless\ifeof\myread\fi{%
  \advance\linecnt by \@ne
  \readline\myread t\expandafter o\csname line-\number\linecnt\endcsname
}

Answer 2

What you are asking about here is "random access" to the lines of text file, which is not supported by any filesystem that I know of. Most programming languages have a seek function that can "jump" forward and backwards in a file by a given number of bytes. This works well in binary files or text files where every line is the same length since the byte offset of "jump to position x" can be calculated.

Unfortunately, most text files have a varying number of bytes per line so it is impossible to predict the offset at which a given line begins without reading the contents of the file. To get around this problem, you need to "index" the file in some way such that lines can be located in an arbitrary order. Two methods that I am aware of are:

• Read the contents of the file, split it into lines and store the results in an array. The array index then provides random access to the file contents. This works well for small files.

• For large files, you have to worry about running out of time or memory. A common trick to conserve memory is to scan the file and store the byte offset of each line in an array rather that the contents of the line its self. These offsets can then be fed to seek in order to locate a line in the file. Perl's Tie::File module uses this method along with some other tricks to represent large files as arrays.

---

Suppose we have the following input file:

foo
bar
baz
bim

With TeX you have two problems:

1. You can only read the file one line at a time---no fancy tricks such as a seek method that lets you jump over content.

2. Without the help of packages, no nice object that can be used as an array with operations like indexed lookup or maximum length.

Solution: Use LuaTeX

This is the easiest way to go since Lua tables make great array objects:

\documentclass{minimal}

\def\readfile#1#2{%
  % Read the contents of a file #1 to an array and store the array name in #2
  \directlua{%
    local input = io.open('#1', 'r')
    #2 = {}
    for line in input:lines() do
      table.insert(#2, line)
    end
    input:close()
  }%
}

\begin{document}   
% Read the contents of io.in to foo
\readfile{io.in}{foo}

% Loop over foo any way you want. The Lua shortcut for table.getn(foo) is #foo,
% but the # character is also special to TeX which causes some confusion.
\directlua{%
  texio.write_nl("Normal:")%
  for i = 1, table.getn(foo) do texio.write_nl(foo[i]) end}

\directlua{%
  texio.write_nl("Reversed:")%
  for i = table.getn(foo), 1, -1 do texio.write_nl(foo[i]) end}

\directlua{%
  texio.write_nl("Odd entries only:")%
  for i = 1, table.getn(foo), 2 do texio.write_nl(foo[i]) end}

\end{document}

The lualatex log output is:

Normal:
foo
bar
baz
bim
Reversed:
bim
baz
bar
foo
Odd entries only:
foo
baz

Lua FILE objects such as input also have a seek method that can be used to pull off advanced tricks.

Answer 3

TeX is able to read lines from a file as long as they have balanced braces. This is accomplished through \openin and \read.

\newread\myread
\openin\myread=file.dat
\read\myread to \command

Now \command will expand to the contents of the first line in file.dat and a subsequent \read\myread to \command will do the same with the following line. The end of the file can be tested with \ifeof\myread, but one should always keep in mind that TeX appends an empty line to each file. The symbolic name \myread can be reused after having closed the file with \closein\myread.

It must be mentioned the possibility of calling, via "shell-escape", external utilities that may be faster and more flexible.