7

Using the code in the answer to How to create form letters from spreadsheet data in ConTeXt?, I am building a document from data in a CSV file. I found that some part of the code is incompatible with chapter titles, such that, if I add a table of contents using \completecontent or add headers using \getmarkings{chapter}, it will not compile.

% macros=mkvi

\unprotect
\startluacode
  local datasets = { }

  local buffersraw   = buffers.raw
  local context      = context
  local ioloaddata   = io.loaddata
  local lpegmatch    = lpeg.match
  local stringformat = string.format
  local stringmatch  = string.match
  local stringsub    = string.sub
  local tableconcat  = table.concat
  local tableswapped = table.swapped

  local die = function (msg) print(msg or "ERROR") os.exit(1) end

  local csv_parser
  do
    --- This is (more than) an RFC 4180 parser.
    --- http://tools.ietf.org/html/rfc4180
    local C, Cg, Cs, Ct, P, S, V
        = lpeg.C, lpeg.Cg, lpeg.Cs, lpeg.Ct, lpeg.P, lpeg.S, lpeg.V

    local backslash = P[[\letterbackslash]]
    local comma     = ","
    local dquote    = P[["]]
    local eol       = S"\n\r"^1
    local noquote   = 1 - dquote
    local unescape  = function (s) return stringsub(s, 2) end
    csv_parser = P{
      "file",
      file    = Ct((V"header" * eol)^-1 * V"records"),
      header  = Cg(Ct(V"name" * (comma * V"name")^0), "header"),
      records = V"record" * (eol * V"record")^0 * eol^0,
      record  = Ct(V"field" * (comma * V"field")^0),
      name    = V"field",
      field   = V"escaped" + V"non_escaped",
      --- Deviate from rfc: the “textdata” terminal was defined only
      --- for 7bit ASCII. Also, any character may occur in a quoted
      --- field as long as it is escaped with a backslash. (\TEX          --- macros start with two backslashes.)
      escaped     = dquote
                  * Cs(((backslash * 1 / unescape) + noquote)^0)
                  * dquote
                  ,
      non_escaped = C((1 - dquote - eol - comma)^0),
    }
  end

  local process = function (id, raw)
    --- buffers may have trailing EOLs
    raw = stringmatch(raw, "^[\n\r]*(.-)[\n\r]*$")
    local data = lpegmatch(csv_parser, raw)
    --- map column name -> column nr
    data.header = tableswapped(data.header)
    datasets[id] = data
  end

  --- escaping hell ahead, please ignore.
  local s_item = [[
  \bgroup
    \string\def\string\insert{\string\getvalue{csv_insert_field}{%s}{%s}}%%
%s%% template
  \egroup
]]

  local typeset = function (id, template)
    local data   = datasets[id] or die("ERROR unknown dataset: " .. id)
    template     = stringmatch(buffersraw(template), "^[\n\r]*(.-)[\n\r]*$")
    local result = { }
    local last = \letterhash data
    for i=1, last do
      result[i] = stringformat(s_item, id, i, template)
    end
    context(tableconcat(result))
  end

  local insert = function (id, n, field)
    local this = datasets[id]
    context(this[n][this.header[field]])
  end

  commands.process_csv      = process
  commands.process_csv_file = function (id, fname)
    process(id, ioloaddata(fname, true))
  end
  commands.typeset_csv_job  = typeset
  commands.insert_csv_field = insert

\stopluacode

\startinterface all
  \setinterfaceconstant{template}{template}
  \setinterfaceconstant    {data}{data}
\stopinterface

\def\processcsvbuffer[#id][#buf]{%
  \ctxcommand{process_csv([[#id]], buffers.raw(\!!bs#buf\!!es))}%
}

\def\processcsvfile[#id][#filename]{%
  \ctxcommand{process_csv_file([[#id]], \!!bs\detokenize{#filename}\!!es)}%
}

%% modeled after \startbuffer
\setuvalue{\e!start csvtemplate}{%
  \begingroup
  \obeylines
  \dosingleempty\csv_template_start%
}

\def\csv_template_start[#id]{%
  \buff_start_indeed{}{#id}{\e!start csvtemplate}{\e!stop csvtemplate}%
}

\installnamespace                  {csvjob}
\installcommandhandler \????csvjob {csvjob} \????csvjob

\appendtoks
  \setuevalue{\currentcsvjob}{\csv_job_direct[\currentcsvjob]}
\to \everydefinecsvjob

\unexpanded\def\csv_job_direct[#id]{%
  \edef\currentcsvjob{#id}%
  \dosingleempty\csv_job_indeed%
}

\def\csv_job_indeed[#setups]{%
  \iffirstargument\setupcurrentcsvjob[#setups]\fi
  \ctxcommand{typeset_csv_job(
                [[\csvjobparameter\c!data]],
                [[\csvjobparameter\c!template]])}%
}

\def\csv_insert_field#id#n[#field]{%
  \ctxcommand{insert_csv_field([[#id]], #n, [[#field]])}%
}

\protect

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                               demo
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% Stepwise instructions.
%% step 1: Define template.
\startcsvtemplate [tpl]
\chapter{\insert[Name]}

You owe \insert[Amount]. Please send it before \insert[Date].
\par
\stopcsvtemplate

%% step 2: Define an input (CSV).
\startbuffer[csdata]
Name,Amount,Date
"Mr. White","\\letterdollar 300","Dec. 2, 1911"
"Mr. Brown","\\letterdollar 300","Dec. 3, 1911"
"Ms. Premise","\\letterdollar 42","Dec. 4, 1911"
"Ms. Conclusion","\\letterdollar 23","Dec. 5, 1911"
\stopbuffer

%% step 3: Parse and store the input.
\processcsvbuffer[one][csdata]
%\processcsvfile[two][test.csv]

%% step 4: Declare a job, joining dataset and template.
\definecsvjob [testing] [
  data=one,
  template=tpl,
]

%% step 5: Enjoy!
\starttext 
  \completecontent
  \testing
\stoptext

The above example contains a table of contents, but it will not compile. If I delete \completecontent, the document compiles fine.

How can I add a table of contents and headers to my document when it is built from this CSV form letter code?

6

Adding \expanded around the \chapter{...} call solves your problem.

Inspecting the .tuc file shows that chapter titles are written to the chapter list (which is a Lua table) as "\\insert [Name]", i.e. as the unexpanded input. \expanded{...} expands its contents (deepest-first); now, \chapter only sees the far less problematic Ms. Premise (for example).

\startcsvtemplate [tpl]
  % \chapter{\insert[Name]}}  %% doesn't work
  \expanded{\chapter{\insert[Name]}}

  You owe \insert[Amount]. Please send it before \insert[Date].
  \par
\stopcsvtemplate

Deeper explanation: expanding \insert[Name] inside the template gets you the desired name. Exporting the same code outside the template (by writing it to a list that gets used elsewhere, and expanding it there) gets you an error.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.