Lualatex luaescapestring and variable expansion

Question

I am trying to write the contents of \foo to a file using LuaLatex. Consider the following example:

\newcommand{\abcd}{test}
\newcommand{\foo}{XXX \abcd}
\directlua{
   local f = {}
   f.foo = token.get_macro 'foo'
(...)

When writing the contents of f.foo to a file later, the content will contain the string XXX \\abcd. If instead we change this to the following:

   f.foo = "\luaescapestring{\foo}"

The result will be the intended XXX test.

When the command \foo contains 'special' commands like ~, the result will be XXX \\protect \\unhbox \\voidb@x \\penalty \\@M \\ {} test. Now this is ugly, but can be parsed/is workable. However, when commands like \\ or \xspace are used, the compilation will fail, such as:

! Undefined control sequence.
\GenericError  ...                                
                                                    #4  \errhelp \@err@     ...

l.180 }

The error message seems to differ based on the command that was used.

Is there a way to expand the command \foo such that recursive expansion is performed and arbitrary special commands are protected/escaped or at the least compilation does not break?

EDIT 1

File main.tex:

\documentclass[british]{report}
\usepackage{luacode}

\begin{document}
Test text
\newcommand{\abcd}{test}
\newcommand{\foo}{XXX \\ \abcd}
\directlua{
    local f = {}
    f.foo = "\luaescapestring{\foo}"
    local fd = io.open("test.out", "a")
    fd:write(f.foo)
    fd:flush()
    fd:close()
}
\end{document}

Compiled with latexmk -pdf -pdflatex="lualatex" main.tex

Above throws an error, but works with \newcommand{\foo}{XXX \abcd}.

Answer 1

In order to make "fragile commands safe in a moving argument" LaTex sets \protect to a suitable value, here the expansion happens while passing to Lua, but is otherwise similar to the expansion that happens in a TeX \write \string is a suitable value here so

\documentclass[british]{report}


\begin{document}
Test text
\newcommand{\abcd}{test}
\newcommand{\foo}{XXX \\ \abcd}

{\let\protect\string
\directlua{
    local f = {}
    f.foo = "\luaescapestring{\foo}"
    local fd = io.open("test.out", "a")
    fd:write(f.foo)
    fd:flush()
    fd:close()
}}

\end{document}

which writes a file with content

XXX \\ test