# Programming with pgf arrays : how to create an array?

I am trying to do some automatic drawing in tikz. The point is I want to be able to provide the minimal quantity of information to my macro, and still have it work.

More precisely (but without entering useless details), I want to take an array of coordinates, perform some computations on them, and draw the picture based on the result of these computations.

But whatever route I try to take, I face the same problem : I need to work with arrays of coordinates. Thanks to \foreach and to the \myarray[index] syntax, some things are (more or less) easy to do with such arrays. One good example is this question.

However, as soon as you try to do more complicated stuff, difficulties arise. For example, how to sort an array? How to map a function on each element of an array?

It seems it would all be easy to do, if we were able to easily create and/or modify an array. However, with the commands I know, it is only possible to read the array's content.

So I have tried concentrating on the most easy task I can imagine yet not do : appending an element to the end of an array.

Since I couldn't find any pure pgf way of doing it, I have tried some low-level hacks (which you can find below for the most part), but pgf's syntax giving importance to braces makes it a hell to track the expansion process, let alone fix it.

\documentclass{article}

\usepackage{tikz}

\newcommand{\printarray}[1]{\noindent
\foreach \i in #1 {\i\\}
}

\begin{document}

\def\myarraywithonebrace{1,2,3,4}
\def\myarraywithtwobraces{{1,2,3,4}}

\printarray{\myarraywithonebrace}
\printarray{\myarraywithtwobraces}

% Ok... So according to the manual, it's supposed to work better with two braces,
% but then what am I doing wrong.

% Whatever, let's append an element
\def\mynewarraywithonebrace{\myarraywithonebrace,5}
\def\mynewarraywithtwobraces{\myarraywithtwobraces,5}

\printarray{\mynewarraywithonebrace}
\printarray{\mynewarraywithtwobraces}

% Ok, so it fails in both cases because it takes the array as a whole.
% Let's try appending sequencially each of its elements, then?

\def\mynewarraywithonebrace{\foreach \i in \myarraywithonebrace {\i, }5}
\def\mynewarraywithtwobraces{\foreach \i in \myarraywithtwobraces {\i, }5}

\printarray{\mynewarraywithonebrace}
\printarray{\mynewarraywithtwobraces}

% Now what... in each case I get a string instead of an array.

% Maybe it means it's time to add that extra pair of braces?

\def\mynewarraywithonebrace{{\foreach \i in \myarraywithonebrace {\i, }5}}
\def\mynewarraywithtwobraces{{\foreach \i in \myarraywithtwobraces {\i, }5}}

\printarray{\mynewarraywithonebrace}
\printarray{\mynewarraywithtwobraces}

% ... Nope

\end{document}


Is my quest hopeless?

-
Is there any requirement for using pure pgf? pgfplotstable would receive your data happily and you can do many things right out of the box. Even better you can attack xparse – percusse Nov 23 '12 at 20:29
No real requirement, no. But the end purpose is to draw a picture. I had looked into pgfplotstable, but it solved only the mapping problem out-of-the-box, as far as I could tell. – T. Verron Nov 23 '12 at 20:33
You can analyze this question: tex.stackexchange.com/questions/54771/… I've implemented something that precisely Christian is complaining about (I didn't even consider efficiency and the code is still wrong waiting for my thesis to be finished. But instead of macros I've dumped them into pgfkeys) but Andrew Stacey made a fantastic attempt using pure LaTeX3 framework and alsolater he has developed it into a package. You can start from there and see if it is worth automating that much. – percusse Nov 23 '12 at 20:40
You are right, there are some gems in there. Maybe for my usage your implementation would be enough, but I'll need to dig into pgfkeys documentation. Andrew's solution is amazing, but sadly, reading and understanding it all, let alone adapting the relevant parts to my needs, would take 10 times as long as writing a simple external script (or even doing the computations by hand), for example. The magic of tex hacking, eh? However, I'll definitely have a deeper look at some of those answers, thanks for the link! – T. Verron Nov 23 '12 at 20:59

I don't think that these PGF arrays are suitable for your task.

You can use an implementation of mine which might be better. But although that implementation proved to be useful, it has quite high demands on TeX skills and has no real support.

In the following, I will elaborate on that implementation. You can read it, and then you should go back and reconsider if you really need arrays in TeX, and then you can use it on your own risk.

But here a foreword: Arrays are some kind of weak point in TeX.

It does not matter what you want to do with arrays, you always run into trouble (and I ran into a lot of it).

TeX has no real builtin support for arrays. And neither has PGF, despite the fact that its math parser can parse such lists.

What you typically expect from an array is fast random access to individual elements and some well-defined handling of the "scope of a variable".

There are a couple of array implementations around. I saw one of them which was written in expl3 some time ago. Their main problem is both their solution for the "scope of a variable" and "fast". They are typically implemented in a similar way to PGF arrays: as a long macro which contains textual separators. Whenever you access or modify the array, you have to touch each element. A simple loop of sorts

for i = 0; i<N; ++i
append to array


typically has O(N^2) time requirement. This holds for both PGF arrays and for the implementation I saw in expl3. The same runtime requirement holds if you have to access N elements in random order. Note: my own package pgfplotstable is a mixture which has fast accumulation, but it also suffers deeply from this problem. It is not a real array implementation, although it suffices for most smaller use-cases.

Eventually, I wrote my own implementation for "real" arrays, i.e. for arrays for which random access takes O(1) cost. Their cost is that they easily blow up the TeX memory because each element is stored in one separate macro. Their cost is also that these arrays cannot be deleted (because they are global) OR they can only exist within some small scope (i.e. until the next \endgroup or closing brace).

It exists only as part of pgfplots and its only documentation is the code comments in pgfplotsarray.code.tex and some test cases.

Here is a copy of my test cases:

\pgfplotsarraynewempty\probe
\pgfplotsarraypushback eins\to\probe
\pgfplotsarraypushback [probe]\to\probe
\pgfplotsarraypushback [probe2]\to\probe
\pgfplotsarraypushback [probe3]\to\probe

Size: \pgfplotsarraysize\probe\to{\count0}\the\count0'

Content:

\pgfplotsarrayforeach\probe\as\curarrayelem{\curarrayelem \par}

\testsubsection{Select}
Test list:

\pgfplotsarraynew\fooarray{Eins\\Zwei\\Drei\\}%
\pgfplotsarrayforeach\fooarray\as\foo{Element \foo\par}%

Getting elements:

\count1=1
Element \the\count1: \pgfplotsarrayselect\count1\of\fooarray\to\elem\elem

Element 2: \pgfplotsarrayselect2\of\fooarray\to\elem\elem

\vskip1cm

\testsubsection{Copy}

Copy:
\pgfplotsarraynew\fooarray{Eins\\Zwei\\Drei\\}%
Source: \pgfplotsarrayforeach\fooarray\as\foo{Element \foo\par}%

\pgfplotsarraycopy\fooarray\to\fooarrayX

Copy:   \pgfplotsarrayforeach\fooarrayX\as\foo{Element \foo\par}%

\testsubsection{reverse iter}
\pgfplotsarrayforeachreversed\probe\as\curarrayelem{\curarrayelem \par}

\testsubsection{reverse iter ungrouped}
\pgfplotsarrayforeachreversedungrouped\probe\as\curarrayelem{\curarrayelem \par}

\testsubsection{sort}
\pgfplotsarraynewempty\testarray
\pgfplotsarraypushback503\to\testarray
\pgfplotsarraypushback087\to\testarray
\pgfplotsarraypushback512\to\testarray
\pgfplotsarraypushback061\to\testarray
\pgfplotsarraypushback908\to\testarray
\pgfplotsarraypushback170\to\testarray
\pgfplotsarraypushback897\to\testarray
\pgfplotsarraypushback275\to\testarray
\pgfplotsarraypushback653\to\testarray
\pgfplotsarraypushback426\to\testarray
\pgfplotsarraypushback154\to\testarray
\pgfplotsarraypushback509\to\testarray
\pgfplotsarraypushback612\to\testarray
\pgfplotsarraypushback677\to\testarray
\pgfplotsarraypushback765\to\testarray
\pgfplotsarraypushback703\to\testarray

Unsorted:

[\pgfplotsarrayforeach\testarray\as\elem{\elem\space}]

\pgfplotsarraysort\testarray

sorted:

[\pgfplotsarrayforeach\testarray\as\elem{\elem\space}]


NOTE: Often, you simply need to accumulate a long list of items and iterate over it ones. That is considerably simpler, even though it still requires a lot of effort to avoid the O(N^2) problem for the accumulation. There are more efficient solutions for such a task.

-
Wow, that's indeed something I'll look at, thanks for that detailed answer. Also, I'd be interested in some examples of the "most efficient solutions for lists" you mentioned at the end. After all, I only used the word array because of the pgfmath paradigm, however what I am looking for is what most languages call list (direct access to the first element, linear access to any element). With some work, I could restrict to considering only lists of less than 5~10 elements, so maybe the extra cost of O(N^2) wouldn't really be restricting. – T. Verron Nov 23 '12 at 20:42
I experimented a lot with "efficient lists". I tried this \pgfplotsarray* stuff that you see in my answer and I tried to use the N^2 stuff suggested in the texbook and I reused some divide-and-conquer-merger which has been implemented for the PGF system layer (which accumulates path parts). The most efficient one was the last one for less than 100000 elements (accumulate, than iterate over it linearly). Since you say that N^2 time would be ok for you, you could take a look at pgfplotstable` and its "Basic level access" section. It might do what you need. – Christian Feuersänger Nov 24 '12 at 11:13