How can one construct a macro that contains a pythontex or fancyvrb environment?

Question

Goal: I want to create a command that executes pythontex code blocks from within a macro.

Application: I have a macro that typesets some equations. I'd like for a side-effect of their use to be the execution of static code within a specified environment.

This problem boils down to addressing "why doesn't verbatim work within...?" for this particular MWE. In this question Geoffrey Poore points out that these environments cannot be used inline, but this doesn't stop me from needing a solution. (e.g., place filecontents* adjacent to the macro, pass the verbatim content of that into a comand that executes it within a session).

\documentclass{minimal}
\usepackage{pythontex}

\newcommand\runsSomeCode[0]{
    \begin{sympycode}
        j = 8
    \end{sympycode}
}

\begin{document}
    \runsSomeCode{}
    \begin{equation}
        j = \sympy{j}
    \end{equation}
\end{document}

Regardless of whether or not a Verbatim or sympycode block is used, the result is the expected:

! FancyVerb Error:
  Extraneous input `j = 8 \end {sympycode} {}' between \begin{sympycode}[<key=val
ue>] and line end
.
\FV@Error ... {FancyVerb Error:
\space \space #1
}

l.11     \runsSomeCode{}

desiderata for this solution

1. Runs the code: I am not looking for a text-formatting solution. I'm not looking for a mono-spaced font or for a code listing. I don't need python output at all.

2. Session-friendly: I have multiple sessions that may or may not require the same code to be executed in their context. Without this requirement, I'd simply create a python function in a sympyblock outside of my macro and invoke it via \sympy{} or \py{}.

3. Must be invoked by a macro: I do not have the luxury of creating a custom environment in the top-level structure of my documents.

4. Must function in math mode

5. Avoid external input files: I build my documents using .latexmkrc, so solutions that remove temporary files at end of compilation are acceptable. using filecontents to generate output is also fair game.

I have no concerns whatsoever if this causes my document to not be able to be processed by depythontex, at this time.

Answer 1

Perhaps you can use xparse's +v-argument-type for having LaTeX read things verbatim.

The shellesc-package provides the command \ShellEscape as a means for writing commands to console/shell.
You can probably use this for calling pythontex "from within the LaTeX-run".

If on my system I save the following code as test.tex

\documentclass{minimal}

\usepackage{shellesc}
\ShellEscape{pythontex \jobname.pytxcode }

\usepackage{amsmath}
\usepackage{xparse}
\usepackage[keeptemps, gobble=auto]{pythontex}

\newcommand\runSomeCodeCatcodeBegingroup{%
  \begingroup
  \catcode`\^^I=12 %
  \catcode`\^^M=12 %
  \newlinechar=\endlinechar
}%
\NewDocumentCommand{\runSomeCode}{}{%
  \runSomeCodeCatcodeBegingroup
  \runSomeCodeInternal
}%
\begingroup
\catcode`\^^M=12\relax%
\NewDocumentCommand{\runSomeCodeInternal}{+v+v}{%
  \endgroup%
  \NewDocumentCommand{\runSomeCodeInternal}{+v+v}{%
     \scantokens{\endgroup#1[##1]^^M##2^^M#2}%
     \ignorespaces%
  }%
}%
\runSomeCodeInternal{\begin{sympycode}}{\end{sympycode}}%

\NewDocumentCommand{\newCodeSnippet}{m}{%
  \runSomeCodeCatcodeBegingroup
  \newCodeSnippetInternal{#1}%
}%

\NewDocumentCommand{\newCodeSnippetInternal}{m+v+v}{%
  \endgroup
  \newcommand*{#1}[1][#2]{\scantokens{\runSomeCode{##1}{#3}}}%
  \ignorespaces
}%

\begin{document}

\newCodeSnippet{\functionExpression}{SessionA}{functionExpression = x**9 + 7*z}

\runSomeCode{SessionA}{x, z = symbols('x z')}
\functionExpression

\runSomeCode{SessionB}{
  x = 2
  z = 3
}
\functionExpression[SessionB]


\begin{equation*}
\sympy[SessionA]{x} = \sympy[SessionB]{x};
\sympy[SessionA]{z} = \sympy[SessionB]{z}
\to \frac{d}{dz} f\left(z\right) = \sympy[SessionA]{functionExpression} = \sympy[SessionB]{functionExpression}
\end{equation*}


\runSomeCode{SessionC}{
  x = 1
  z = 1
}
\functionExpression[SessionC]

\begin{equation*}
\sympy[SessionA]{x} = \sympy[SessionC]{x};
\sympy[SessionA]{z} = \sympy[SessionC]{z}
\to \frac{d}{dz} f\left(z\right)  = \sympy[SessionA]{functionExpression} = \sympy[SessionC]{functionExpression}
\end{equation*}


\end{document}

and compile it twice via the shell-command

pdflatex -shell-escape test.tex

, then I get this:

Answer 2

I have found a solution that works with both fancyvrb and pythontex. This solution as applied to fancyvrb is actually found within the documentation of the filecontentsdef package. Hopefully this will help others with what seems to be a FAQ.

\usepackage[keeptemps, gobble=auto]{pythontex}
\usepackage{xparse}
\usepackage{filecontentsdef}

% [1] store tabs as active characters so they can be handled by fancyvrb
{\catcode`\^^I=\active\gdef\FCDtabtomacro{\noexpand^^I}}

% [2]
\begin{filecontentsdefmacro}{\testwe}
    x, z = symbols('x z')
    functionExpression = x**9 + 7*z
\end{filecontentsdefmacro}

% Define our macro that should execute the verbatim code block
\NewDocumentCommand\nestedcode{m}{{% [3]
    \renewcommand*\FCDprintenvname{sympycode}% [4]
    \renewcommand*\FCDprintenvoptions{% [5] environment options can be set
        \unexpanded{[#1]}%
    }%
    \filecontentsprint\testwe % [6] we could pass a csname here
    \frac{d}{dz} f\left(z\right) = \sympy[#1]{functionExpression}
}}

The steps to apply this solution within your own document are as follows:

1. Per the filecontentsdef package, we save tab as an active character.
2. Define your content via filecontentsdef macro facilities. These macros will be used as identifiers later for code that we wish to execute.
3. Construct any macro that you wish. I utilized xparse here in order to stress this code within a "complicated" calling hierarchy (if that has any effect). If you wish to prevent polluting definitions then wrap the macro body in a group per normal LaTeX conventions.
4. Set the environment that you wish the saved macro to be expanded into
5. Set any optional arguments accepted by that environment (such as the session name for sympy)
6. Invoke \filecontentsprint to expand the saved content in the desired context.

To demonstrate the desired behavior, the following document:

\begin{document}
    An expression, nested within an equasion environment, producing text while
    modifying a session whose name was passed by argument:
    \begin{equation}
        \nestedcode{sessionname}
    \end{equation}

    Confirming that the session does, in fact, subsequently contain those symbols:
    \begin{equation}
        \sympy[sessionname]{x}
    \end{equation}
\end{document}

Yields the desired output.

Answer 3

It's clear that you have to supply the characters with the correct catcodes, which can be done by the following ways:

• The proper way: Lookup the package's documentation to see if there's any command that works without changing catcode.

  For verbatim there's \texttt, for pythontex there's \py and \pyc, for listings there's \lstinline.

  See also:

  • newcommand for verbatim
  • Verbatim inside a command

• The manual way: You have to somehow figure out which catcodes are expected by the environment, then manually convert the characters into the correct catcode and feed it to the environment.

• The automatic way equivalent (old engine): Write the content to an external file, \input the file, then (optional) delete the file.

• The automatic way: Make a token list with suitable catcodes, then \scantokens the result. (used in the answer below)

There's no other way, because in order to know what catcode is correct, you have to run the code, and once you've executed the code you cannot "undo" the effect.

---

The solution below uses the \scantokens approach, which is mostly the same approach as the other answer... but hopefully easier to understand.

\documentclass{article}
\usepackage{fancyvrb}
\begin{document}
\ExplSyntaxOn
% ======== Define \my__tl_set_verbatim : similar to \tl_set:Nn, but second argument is of xparse's +v type. ========
\NewDocumentCommand \my__tl_set_verbatim {m +v} {
    \tl_set:Nn #1 {#2}
}
% ======== Define some auxiliary helper TLs. ========
\tl_set:Nx \my__newline_char_other_cat   {\char_generate:nn {`\^^M} {12}}  % A newline character with "other" category code.
\my__tl_set_verbatim \my__begin_verbatim {\begin{Verbatim}[frame=single, numbers=left]}
\my__tl_set_verbatim \my__end_verbatim   {\end{Verbatim}}
% ======== Define the main command. ========
\cs_set:Npn \my__command:n #1 {
    \exp_args:Nx \scantokens {
        \my__begin_verbatim     \my__newline_char_other_cat
            x~ =~ #1            \my__newline_char_other_cat
        \my__end_verbatim       \my__newline_char_other_cat
    }
}
% ======== Use that command. ========
\my__command:n {123}
\ExplSyntaxOff
\end{document}

This solution uses fancyvrb's Verbatim environment for demonstration, but it should work with other environments.

The result is:

Of course, to use the commands outside of \ExplSyntaxOn mode, define an appropriately-named command.

---

Useful tips:

• For debugging, change the command definition to

  \cs_set:Npn \my__command:n #1 {
      \tl_set:Nx \my__tmp {
         \my__begin_verbatim     \my__newline_char_other_cat
            x~ =~ #1             \my__newline_char_other_cat
         \my__end_verbatim       \my__newline_char_other_cat
      }
      \tl_analysis_show:N \my__tmp
      \exp_args:NV \scantokens {
         \my__tmp
      }
  }
  

  Lookup documentation of \tl_analysis_show:N for more information.

• Things that will not work:

  • Use \detokenize{\end{Verbatim}}. Because the environment expects \end{Verbatim} exactly, not \end {Verbatim} (with a space between \end and {) to end the environment.

  • Include the newline in the \my__begin_verbatim definition line this:

    \my__tl_set_verbatim \my__begin_verbatim {\begin{Verbatim}[frame=single, numbers=left]
    }
    

    According to my other answer xparse's +v argument will convert newline to space character in \ExplSyntaxOn mode. (however, switching to \ExplSyntaxOff before the definition works)