How to you properly specify an alternative to the pygmentize external command for minted?

Question

A while ago I asked how to extend minted to colour-highlight my chosen key words. The consensus was "make the changes in the external pygment program" -- so I did.

The changes are small and the output of my modified pygment program seems to be fine. If I install my new pygment module system-wide, my document compiles correctly and the changes to syntax highlighting are as expected.

The catch is I don't want to install my replacement pygment module system-wide, that's too invasive and I want to be able to distribute my document for compilation on other people's computers. I have a simple bash script bin/pygmentize.sh which invokes the new version of pygmentize as a set of local scripts. This bash script is a tested, functioning drop-in replacement for the global pygmentize program. Moreover, when it is invoked via latex/minted in the mannar shown below, it generates the correct markup, but the document fails to compile.

Here's a comparison of the markup that my altered pygmentize generates versus the original:

Before:

\begin{Verbatim}[commandchars=\\\{\}]
\PYG{c+cp}{\PYGZsh{}include} \PYG{c+cpf}{\PYGZlt{}stdio.h\PYGZgt{}}

\PYG{k+kt}{void} \PYG{n+nf}{main} \PYG{p}{()}
\PYG{p}{\PYGZob{}}
    \PYG{n+nf}{printf} \PYG{p}{(}\PYG{l+s}{\PYGZdq{}Hello, world!\PYGZdq{}}\PYG{p}{);}
\PYG{p}{\PYGZcb{}}
\end{Verbatim}

After:

\begin{Verbatim}[commandchars=\\\{\}]
\PYG{c+cp}{\PYGZsh{}include} \PYG{c+cpf}{\PYGZlt{}stdio.h\PYGZgt{}}

\PYG{k+kt}{void} \PYG{n+nf}{main} \PYG{p}{()}
\PYG{p}{\PYGZob{}}
    \PYG{n}{printf} \PYG{p}{(}\PYG{l+s}{\PYGZdq{}Hello, world!\PYGZdq{}}\PYG{p}{);}
\PYG{p}{\PYGZcb{}}
\end{Verbatim}

This code was taken from the minted cache directory in both cases, it was generated through latex's invocation of pygmentize, not my own manual invocation. (I deleted the minted cache, switched to using my updated pygment, and ran pdflatex again.)

The only difference between the generated markup is that \PYG{n}{printf} becomes \PYG{n+nf}{printf} so I would expect that the rest of the document would compile the same way. Not so

(/tmp/test/minted-cache/D855E0EC6A86300E2FD8FEE675873CCC2C2645CE96B6E2A989A9815
F3192183A.pygtex
! Undefined control sequence.
<argument> \PYG {c+cp}{\PYGZsh 
                               {}include} \PYG {c+cpf}{\PYGZlt {}stdio.h\PYG...
l.2 ...ude} \PYG{c+cpf}{\PYGZlt{}stdio.h\PYGZgt{}}

!  ==> Fatal error occurred, no output PDF file produced!

This error occurs when compiling the above piece of markup. If I am reading this correctly, it is the \PYGZsh command which is not recognised. This error strikes me as very odd -- minted is clearly loading and running, the earlier \PYG command was recognised as a control sequence, and the error manifests in a section of code which has not changed.

Here's how I got my altered pygmentize to be invoked: poking around in the minted source I saw this:

\ifcsname MintedPygmentize\endcsname\else
  \newcommand{\MintedPygmentize}{pygmentize}
\fi

Then a little later

\MintedPygmentize\space -l #2 -f latex -P commandprefix=PYG -F tokenmerge (...etc)

It seems that by defining or redefining \MintedPygmentize I should be able to invoke a different program.

I tried doing this before importing minted

\newcommand{\MintedPygmentize}{bash bin/pygmentize.sh}

and I tried doing this after importing minted

\renewcommand{\MintedPygmentize}{bash bin/pygmentize.sh}

Either way round produces the error. I have confirmed that my bin/pygmentize.sh is invoked, and that it is receiving its inputs and producing its outputs.

I have concluded that redefining \MintedPygmentize has caused some black magic side effects.

So, in order to have a portably-compilable document, how do I tell the minted package to use my external script to generate the markup in a way that doesn't cause strange errors like this to appear?

Answer 1

The reason why \renewcommand{\MintedPygmentize}{bash bin/pygmentize.sh} is not working for you is due to how the pygmentize check is implemented in minted.

This line is what executes the check on non-windows systems (e.g., linux, macOS):

    \immediate\write18{which #1 && touch \jobname.aex}
    \IfFileExists{\jobname.aex}
     {\AppExiststrue
      \DeleteFile{\jobname.aex}}
     {\AppExistsfalse}
  \fi}

Your custom command then gets expanded to:

which bash bin/pygmentize.sh && touch \jobname.aex

Which finds bash, but not bin/pygmentize.sh. The solution is to either make bin/pygmentize.sh executable (chmod u+x bin/pygmentize.sh) so that which does not return an error or to bypass the check completely. You can bypass the check by declaring your command as:

\renewcommand{\MintedPygmentize}{which ; bash bin/pygmentize.sh}

This will get expanded to: which which; bash bin/pygmentize.sh && touch \jobname.aex, and will thus pass the check. On actual invocations, which ; does not produce any output so the whole command invocation will behave as if this part of the command is not there.