How to read a file line by line and store each line into an array?

Question

As said in the title, I am looking for a way to read a file line by line (done!) and store each line, each string, in an array.

I have searched the web and stackexchange for a while but I cannot find a suitable solution---in fact, I still do not know, whether arrays as data type do exist or not.

I have no MWE, yet, because of the lack of information.

\newread\file
    \openin\file=myfilename.txt
    \loop\unless\ifeof\file
        \read\file to\fileline % Reads a line of the file into \fileline
            \fileline
    \repeat
\closein\file

This is my construct of reading a file line by line, which seems to work properly. \fileline is the string I want to store in an array or somewhat equal to an array. In Java I would use an ArrayList;

ArrayList<String> list = new ArrayList<String>();

would be the object and

list.add(readline);

would be the statement to store the parameter readline into list.

---

Ok, well. I fear those solutions are much too complicated for me to understand, in fact, I have no idea what is going on in all of your answers. My problem, the solution, respectively, seems to be way too difficult.

I would like to read a file which contains exactly one character per line and then store it in an array or whatever type of list, so that I will be able to read the array with myarray[2] or something to get the second (or third, depending on whether to start at 1 or 0) value from the list to do things with it.

I will examine the datatool package further---at first sight it seems vaguely to do what I want, but I do not know how, yet.

Thanks for all of your replies and I am sorry to have wasted your time!

Regards hringriin

EDIT:

Ok, well. I fear those solutions are much too complicated for me to understand, in fact, I have no idea what is going on in all of your answers. My problem, the solution, respectively, seems to be way too difficult.

I would like to read a file whichs contains exactly one character per line and then store it in an array or whatever type of list, so that I will be able to read the array with myarray[2] or something to get the second (or third, depending on whether to start at 1 or 0) value from the list to do things with it. So my problem itself is the way of storing the data or the chars from the file in LaTeX. The problem is that I cannot read a single line from a file.

I will examine the datatool package further---at first sight it seems vaguely to do what I want, but I do not know how, yet.

Answer 1

It's not really difficult with expl3. I define a command \ReadFile that takes two arguments

\ReadFile{\myarray}{somefile.dat}

The first argument is a control sequence name, the second is the file to read from. With this, the file will be read in and the command \myarray defined so that

\myarray{2}

yields the second item; the special call \myarray{*} will return the number of items. You can also call \myarray{-1} to access the last item.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\ior_new:N \g_hringriin_file_stream

\NewDocumentCommand{\ReadFile}{mm}
 {
  \hringriin_read_file:nn { #1 } { #2 }
  \cs_new:Npn #1 ##1
   {
    \str_if_eq:nnTF { ##1 } { * }
      { \seq_count:c { g_hringriin_file_ \cs_to_str:N #1 _seq } }
      { \seq_item:cn { g_hringriin_file_ \cs_to_str:N #1 _seq } { ##1 } }
   }
 }

\cs_new_protected:Nn \hringriin_read_file:nn
 {
  \ior_open:Nn \g_hringriin_file_stream { #2 }
  \seq_gclear_new:c { g_hringriin_file_ \cs_to_str:N #1 _seq }
  \ior_map_inline:Nn \g_hringriin_file_stream
   {
    \seq_gput_right:cx 
     { g_hringriin_file_ \cs_to_str:N #1 _seq }
     { \tl_trim_spaces:n { ##1 } }
   }
  \ior_close:N \g_hringriin_file_stream
 }

\ExplSyntaxOff

\begin{document}

\ReadFile{\myarray}{somearray.dat}

\myarray{*}

\myarray{1}

\myarray{2}

\myarray{3}

\myarray{-1}

\myarray{-2}

\myarray{-3}

\end{document}

If the file somearray.dat is

And now for something completely different 
1 2 3 4 5 6 7 8
a bc def ghij

(I used the same as Christian), the result will be

Answer 2

Latex doesn't have any particularly good array data type -- in particular, in a normal language, you would expect to have some constant time operations, some linear time. In Latex, insertion at the beginning or end and concatenation are perhaps constant time, but all other operations are done by mapping some function over the list, and so are linear time.

The standard LaTeX array looks like \def\myarray{\\{entry1}\\{entry2}\\{entry3}...}. This is good for concatenation and mapping. You map another function over it by saying \def\\#1{do something with #1} \myarray.

If you want to store the lines like this, what you would say is:

\newtoks\temptoksi
\newtoks\temptoksii
\def\myarray{}

\newread\file
    \openin\file=myfilename.txt
    \loop\unless\ifeof\file
        \read\file to\fileline % Reads a line of the file into \fileline
        \temptoksi\expandafter{\myarray}
        \temptoksii\expandafter{\fileline}
        \edef\myarray{\the\temptoksi\\{\the\temptoksii}}
    \repeat
\closein\file

The way this works is that \edef recursively expands the body of its definition before setting \myarray equal to the result. A token list is somewhat like a macro with no arguments, except you call it by saying \the\tokenlist, and the contents of the token list aren't recursively expanded in \edef. So the three lines I added to the body of the loop say "set \temptoksi equal to the contents of \myarray, set \temptoksii equal to the contents of \fileline, then set \myarray equal to "old contents of myarray" \\{"contents of fileline"}. Of course for readability you could make a separate definition:

\def\addtomacro#1#2{
    \temptoksi\expandafter{#1}\temptoksii\expandafter{#2}
    \edef#1{\the\temptoksi\\{\the\temptoksii}
}

and then you can just say \addtomacro\myarray\fileline.

Edit: Actually there is a way to implement better arrays but it's more technical. Basically, the idea is to store each element of the array in a different control sequence, so the first element would be stored in a control sequence named "myarray1," the second in "myarray2," etc. This is essentially a hash array because latex looks up the definition of a control sequence in an internal hash table. To use a very minimal version of this sort of array, you would say something like:

\def\setarrayelt#1#2{\expandafter\xdef\csname array@#1@#2\endcsname}
\def\getarrayelt#1#2{\csname array@#1@#2\endcsname}

Then you can say:

\newcount\arraylength
\arraylen=0
\newread\file
    \openin\file=myfilename.txt
    \loop\unless\ifeof\file
        \read\file to\fileline % Reads a line of the file into \fileline
        \setarrayelt{myarray}{\the\mycount}{\fileline}
        \advance\arraylength1
    \repeat
\closein\file

Of course, you can do more sophisticated things, like store the length in \array@myarray@length and then make \getarrayelt throw an out of bounds error if you give it an argument that's too large (or not a number). You could then add macros like \addelttoendofarray, etc.

Answer 3

Perhaps the readarray package can be of help. EDITED to use newer preferred syntax from 2016-11-07 package update.

\documentclass{article}
\usepackage{readarray}[2016-11-07]
\usepackage{filecontents}
\begin{filecontents*}{mydata.dat}
First record
2nd record
3rd record
...
last record (before blank record at end)
\end{filecontents*}
\begin{document}
\readrecordarray{mydata.dat}\Mydata
There are \MydataROWS{} rows of data\par
The 5th record is: \Mydata[5]\par
The 3rd record is: \Mydata[3]
\end{document}

Answer 4

Here is a poor man's no package way expanding upon your loop. A few features are illustrated by the result:

1. a "line" must have balanced braces,

2. commented out things will disappear,

3. \par tokens are inserted.

More robust methods will input the file verbatim first. See also below the method with etex's \readline.

% http://tex.stackexchange.com/questions/285451/how-to-read-a-file-line-by-line-and-store-each-line-into-an-array
% Time-stamp: <01-01-2016 18:19:49 CET>
\documentclass[a4paper]{article}
\usepackage[margin=3cm]{geometry}
% hringriin
\newread\MyFile

\makeatletter
\newcommand*\FileToArray [1]{%
    \edef\rii@tempb {#1}%
    \openin\MyFile=\rii@tempb\relax
    \def\rii@tempc {0}%
    \loop
    \unless\ifeof\MyFile
        \edef\rii@tempc {\the\numexpr \rii@tempc+\@ne}%
        \read\MyFile to \rii@tempa
        \expandafter\let 
        \csname \[email protected]\rii@tempc\endcsname\rii@tempa
    \repeat
    \closein\MyFile
    \expandafter\let\csname \[email protected]\endcsname\rii@tempc
    \wlog{Loaded \rii@tempc\space lines from \rii@tempb.}% 
}

\newcommand*\FileLine [2]{%
    \csname #1.line\the\numexpr#2\endcsname
}%

\newcommand*\FileLineCount [1]{%
    \csname #1.linecount\endcsname
}%

\newcommand*\VerbatimFileLine [2]{%
    \detokenize\expandafter\expandafter\expandafter 
               {\csname #1.line\the\numexpr#2\endcsname}%
}%


\makeatother

\begin{document}\thispagestyle{empty}

\ttfamily

\FileToArray {\jobname}

\count255 0
\loop
\advance\count 255 1
\ifnum\count255> \FileLineCount{\jobname}
\else
\the\count255: \VerbatimFileLine {\jobname}{\count255}\par
\repeat

\end{document}

Output:

About the same with \readline (texdoc etex). As \readline also catches the endline characters, we play tricks to get rid of it. In this second code, no \VerbatimFileLine as \FileLine already produces "verbatim" output.

% http://tex.stackexchange.com/a/285459/4686
% Time-stamp: <01-01-2016 18:36:46 CET>
\documentclass[a4paper]{article}
\usepackage[margin=3cm]{geometry}
% hringriin
\newread\MyFile

\makeatletter
\newcommand*\FileToArray [1]{%
    \edef\rii@tempb {#1}%
    \openin\MyFile=\rii@tempb\relax
    \def\rii@tempc {0}%
    \loop
    \unless\ifeof\MyFile
        \edef\rii@tempc {\the\numexpr \rii@tempc+\@ne}%
        \readline\MyFile to \rii@tempa
        \expandafter\let 
        \csname \[email protected]\rii@tempc\endcsname\rii@tempa
    \repeat
    \closein\MyFile
    \expandafter\let\csname \[email protected]\endcsname\rii@tempc
    \wlog{Loaded \rii@tempc\space lines from \rii@tempb.}% 
}

\def\rii@@cleanendline #1{\def\rii@cleanendline ##1#1{##1}}
\begingroup
\lccode`. 13
\lowercase{\endgroup\rii@@cleanendline.}

\newcommand*\FileLine [2]{%
    \expandafter\expandafter\expandafter\rii@cleanendline
    \csname #1.line\the\numexpr#2\endcsname
}%


\newcommand*\FileLineCount [1]{%
    \csname #1.linecount\endcsname
}%

\makeatother

\begin{document}\thispagestyle{empty}

\ttfamily

\FileToArray {\jobname}

\count255 0
\loop
\advance\count 255 1
\ifnum\count255> \FileLineCount{\jobname}
\else
\the\count255: \FileLine {\jobname}{\count255}\par
\repeat

\end{document}

(there is one more empty line 57: on page 2)

Answer 5

According to the O.P. all solutions here are too much -- well I keep it.

Here's an expl3 - way of reading strings to a seq variable which can be regarded as an array.

The details of what to do with this strings is up to the O.P.

Here's the file somearray.dat

And now for something completely different 
1 2 3 4 5 6 7 8
a bc def ghij

Code

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\ior_new:N \l_input_stream
\seq_new:N \g_all_lines_seq
\bool_new:N \g_not_eof_bool
\ior_open:Nn \l_input_stream {somearray.dat} %Open the file

\newcommand{\ReadOneLine}{%
  \ior_get_str:NN \l_input_stream \l_tmpa_tl  % Read one Line 
  \seq_gput_right:NV \g_all_lines_seq {\l_tmpa_tl} % Put it to next array cell
}

\newcounter{foo}

\newcommand{\ReadAll}{%
  \bool_gset_true:N \g_not_eof_bool
  \bool_while_do:nn {\g_not_eof_bool} {%
    % can be replaced by \ior_if_eof:NTF most likely
    \if_eof:w \l_input_stream
    \bool_gset_false:N \g_not_eof_bool  % Set the end of reading
    \else:
    \ReadOneLine
    \fi:
  } % End of loop
}

\newcommand{\ShowArrayLine}[1]{%
  \seq_item:Nn \g_all_lines_seq {#1}
}

\newcommand{\CloseFile}{%
\ior_close:N \l_input_stream
}

\ExplSyntaxOff

\begin{document}
\ReadAll
\CloseFile

\ShowArrayLine{1}

\ShowArrayLine{3}

\ShowArrayLine{3}

\ShowArrayLine{2}

\end{document}

Answer 6

While (La)TeX is incredibly versatile, and even Turing complete, that doesn't mean trying to do everything in it is a good idea. They are languages designed for a rather narrow use area, you can twist them out of shape and do outside-the-design stuff. But that isn't necessarily wise (or a good use of your time, whether now when writing the solution nor later when tying to understand it to reuse it or fix it).

Today there are very capable general purpose languages with a special knack at manipulating text, like Python or Perl, with very good implementations running everywhere where TeX runs. Consider leveraging the best aspects of different tools.

Answer 7

If you are a R user, an easy solution is Sweave/knitr to read the text file as a data frame of one column, so you can use \Sexpr{} to show any row ... and more:

\documentclass{article}
\usepackage{verbatim} % to show verbatim the external text file
\parindent0pt
\begin{document}
\subsubsection*{%
How to read lines of a external file in \LaTeX\ with R} 
\begin{enumerate}
\item You should have a file with some plain text. 
We will use this  \verb|test.csv| :\par
\dotfill
\verbatiminput{test.csv}  
\dotfill
\item  Let R to read  \verb|test.csv| as the data frame \texttt{x} 
with one column.
<<text, echo=F, comment=NA>>=
x <- read.table("test.csv", header=F,  as.is=TRUE, sep="|")
#x[] <- lapply(x, as.character)
@
\item Now you can insert only within \LaTeX\  the second line, 
that  have  \Sexpr{nchar(x[2,])} characters including spaces:\par  
\texttt{\Sexpr{x[2,]}}\par
\end{enumerate}
\subsubsection*{Procastination nook} 
Write at hand the external text file: \parskip1.5em\par
\begin{enumerate}
\item \rule{\Sexpr{nchar(x[1,])*0.55}em}{1pt} 
(\Sexpr{nchar(x[1,])} characters) \par
\item \rule{\Sexpr{nchar(x[2,])*0.55}em}{1pt}
(\Sexpr{nchar(x[2,])} characters) \par
\item \rule{\Sexpr{nchar(x[3,])*0.55}em}{1pt} 
(\Sexpr{nchar(x[3,])} characters) \par
\item \rule{\Sexpr{nchar(x[4,])*0.55}em}{1pt} 
(\Sexpr{nchar(x[4,])} characters) \par
\end{enumerate}
\end{document}