Does writing to a property list take longer than writing to a token list?

Question

In the question Write a column selectively to the appropriate row using pgfplotstable? I offered the following:

\documentclass{article}

\usepackage{xparse}
\usepackage{booktabs}
\usepackage{longtable}
\usepackage{filecontents}

\begin{filecontents*}{namespgfa.csv}
5501,Kathirvelu A
5502,Gugan K
5503,Kalaitchelvi S
5504,Suresh S
5505,Mahesh K
\end{filecontents*}
%
\begin{filecontents*}{markspgfa.csv}
5501,67
5502,25
5503,62
5505,95
\end{filecontents*}

\ExplSyntaxOn
\tl_new:N \g_tab_rows_tl
\ior_new:N \g_names_ior
\ior_new:N \g_marks_ior
\prop_new:N \g_names_prop

\ior_open:Nn \g_names_ior {namespgfa.csv}
\ior_open:Nn \g_marks_ior {markspgfa.csv}

\cs_new:Npn \set_name_keys:w #1,#2\q_stop
    {
        \prop_put:Nnn \g_names_prop {#1} {#2}
    }

\cs_new:Npn \tab_write_keys:w #1,#2\q_stop 
    {
        \prop_gpop:NnN \g_names_prop {#1} \l_tmpa_tl 
        \tl_gput_right:Nn \g_tab_rows_tl {#1&} 
        \tl_gput_right:NV \g_tab_rows_tl \l_tmpa_tl
        \tl_gput_right:Nn \g_tab_rows_tl {&#2\\} 
    }
%replacing the above two control sequences with the following slows compilation substantially
%\cs_new:Npn \set_name_keys:w #1,#2\q_stop
%   {
%        \prop_put:Nnn \g_names_prop {#1} {#1&#2&}
%   }
%
%\cs_new:Npn \tab_write_keys:w #1,#2\q_stop
%    {
%        \prop_gpop:NnN \g_names_prop {#1} \l_tmpa_tl
%        \tl_gput_right:NV \g_tab_rows_tl \l_tmpa_tl
%        \tl_gput_right:Nn \g_tab_rows_tl {#2\\}
%    }

\ior_str_map_inline:Nn \g_names_ior
    {
        \set_name_keys:w #1\q_stop
    }

\ior_str_map_inline:Nn \g_marks_ior
    {
        \tab_write_keys:w #1\q_stop
    }

\ior_close:N \g_names_ior
\ior_close:N \g_marks_ior

\NewDocumentCommand { \WriteRows } {}
    {
        \tl_use:N \g_tab_rows_tl
    }

\ExplSyntaxOff

\begin{document}

\begin{longtable}{clc}
Reg.No.&Name&Marks\\
\toprule
\WriteRows
\bottomrule
\end{longtable}

\end{document}

Background: The first column of both namespgfa.csv and markspgfa.csv contains id numbers, while the second column contains a corresponding name and mark respectively. The idea was very basic: associate each id with a name, and insert that name between the corresponding id and mark in a table. Since speed was an issue, I thought that the commented changes in the above code might speed things up. My thinking was that on for .csv with n lines, \set_name_keys:w will get called n times regardless, therefore using that cs to write some extra information would save ~n calls to \tl_gput_right:Nn \g_tab_rows_tl {#1&}. However, on test .csv's with 4000 lines, the second method took ~30s while the first took ~25.

My question is: why is there such a substantial difference in compile times?

Here are the test files I used:

1. markspgf http://www.smallfiles.org/download/1451/markspgf.csv.html
2. namespgf http://www.smallfiles.org/download/1452/namespgf.csv.html

To use these, remove the "a" from the filenames in \ior_open:Nn \g_names_ior {namespgfa.csv} and \ior_open:Nn \g_marks_ior {markspgfa.csv}.

Answer 1

Ultimately, expl3's prop data type is constructed using a TeX macro (at the moment: we used to use token registers). When you assign to a prop, the assignment first needs to 'look' for the key before either adding a new key/value pair or replacing the value for an existing key. This is done using a delimited macro at the TeX level. So when you add to a prop, TeX has to read over all of the tokens in the underlying macro, which gets slower as the content gets larger. On the other hand, assigning to a tl (which is also a macro at the TeX level) does not involve a parsing step, so the only size consideration is what you are adding, not what is already there. Thus ultimately anything which makes prop variables larger will slow things down. It's of course a balance (as each assignment takes time), but if you are looking at very large structures and large numbers of tokens then splitting things up may be more efficient.

(There is an experimental 'data table' data type in the team SVN. This shows the same problem, but to a greater extent, and I am intending to revise the code to use more csnames each containing less data, as the balance here is clearly wrong.)