Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

I'd like to know if there's a script that reads a .tex file and replace every instance of a non-standard TeX command with whatever it is replacing. I am not sure if what I want is clear but let me give an example:

Suppose the input is:

\documentclass{amsart} 
\usepackage{amsmath,amssymb}
\newcommand{\N}{\mathbb{N}}
\DeclareMathOperator{\End}{End}

\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's denote this ring by $\End(A)$. Throughout the lecture, $\N$ will denote
the set of natural numbers.
\end{document} 

Then, a desirable output of such a script is:

\documentclass{amsart}
\usepackage{amsmath, amssymb}
\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's denote this ring by $\operatorname{End}(A)$. Throughout the lecture,  
 $\mathbb{N}$ will denote the set of natural numbers.
\end{document}

P.S.: I think I had seen something to this effect but I neither remember the place nor a key word to fire up Google.


I meant to write that, all answers here are awesome, but I miscounted 2 for 4. :(

share|improve this question
1  
You want something like a preprocessor. See here or here for similar questions. –  Juri Robl Aug 18 '12 at 15:56
    
How deep do you want to go? Just one level or completely expanded? –  mrf Aug 18 '12 at 16:25
    
@mrf I'd like to replace, for instance, $\N$ with $\mathbb {N}$. So, basically, I'd like the script to replace foo with oof if the preamble has the line \newcommand{foo}{oof}. –  kan Aug 18 '12 at 16:31
    
Personally I think perl will be ideal for this... but are you after something in TeX? –  cmhughes Aug 18 '12 at 16:45
1  
this may be what you want. But you have to define yourself which macros you want replaced. –  Juri Robl Aug 18 '12 at 16:48

5 Answers 5

Info I've been forced by the TeX.sx chatroom mafia to post my lovely, buggy, terrible, traumatic, post-apocalyptic poor man's implementation of a replacement script. :)

Well, sadly this won't be a TeX answer. :) Here's my humble attempt, with a script language I'm terrible at.

(I'm looking at you, Python!)

import re
import sys

if len(sys.argv) != 3:
    print('We need two arguments.')
    sys.exit()

inputHandler = open(sys.argv[1], 'r')

mathDictionary = {}
commandDictionary = {}

print('Extracting commands...')
for line in inputHandler:
    mathOperator = re.search('\\\\DeclareMathOperator{\\\\([A-Za-z]*)}{(.*)}', line)
    if mathOperator:
        mathDictionary[mathOperator.group(1)] = mathOperator.group(2)
    newCommand = re.search('\\\\newcommand{\\\\([A-Za-z]*)}{(.*)}', line)
    if newCommand:
        commandDictionary[newCommand.group(1)] = newCommand.group(2)

inputHandler.seek(0)

print('Replacing occurrences...')
outputHandler = open(sys.argv[2],'w')
for line in inputHandler:
    current = line
    for x in mathDictionary:
        current = re.sub('\\\\DeclareMathOperator{\\\\' + x + '}{(.*)}', '', current)
        current = re.sub('\\\\' + x + '(?!\w)', '\\operatorname{' + mathDictionary[x] + '}', current)
    for x in commandDictionary:
        current = re.sub('\\\\newcommand{\\\\' + x + '}{(.*)}', '', current)
        current = re.sub('\\\\' + x + '(?!\w)', commandDictionary[x], current)
    outputHandler.write(current)

print('Done.')

inputHandler.close()
outputHandler.close()

Now, we simply call it:

$ python myconverter.py input.tex output.tex
Extracting commands...
Replacing occurrences...
Done.

input.tex

\documentclass{amsart} 
\usepackage{amsmath,amssymb}
\newcommand{\N}{\mathbb{N}}
\DeclareMathOperator{\End}{End}

\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's denote this ring by $\End(A)$. Throughout the lecture, $\N$ will denote
the set of natural numbers.
\end{document} 

output.tex

\documentclass{amsart} 
\usepackage{amsmath,amssymb}



\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's denote this ring by $\operatorname{End}(A)$. Throughout the lecture, $\mathbb{N}$ will denote
the set of natural numbers.
\end{document} 

Limitations:

  • It's my code, so beware! :)
  • It works only with \DeclareMathOperator{...}{...} and \newcommand{...}{...}.
  • No optional arguments are supported for \newcommand.
  • The declaration must be in only one line.
  • Balanced curly brackets, please. :)

I know that regular expressions are not suitable for parsing TeX, but they should work for very simple replacements.

Here's a beautiful reading about regex. Have fun. :)

share|improve this answer
1  
Paulo, you may need to make some of your patterns non-greedy. For example, '{(.*)}' will match all of '{\N}{\color{blue} yes}', which I suspect is not what you want. '(.*?)' is the notation, I think. But even that will get you into trouble with nested braces. Regexen can't express arbitrarily recursive structures, as your link tells... –  Brent.Longborough Aug 18 '12 at 20:44
    
@Brent: oops sorry for the faulty regex. :) I'll take a better look at the Python documentation, maybe I could use make use of a lookahead feature. I trust your Python wisdom. :) –  Paulo Cereda Aug 18 '12 at 20:54
1  
You could also turn {(.*)} into {([^}]*)}, which will do what I think you want, maybe more efficiently. –  Brent.Longborough Aug 18 '12 at 20:58
    
It might be worth considering using Python's raw string literals to cut down on the \ chatter: r'\\DeclareMathOperator{\\([A-Za-z]*)}{(.*)}', for instance. –  jpmc26 Jan 31 at 0:20

By accident I came upon de-macro, which is a Python script for such a purpose. It is included in TeX Live.

Limitation: It only affects \newcommand, \renewcommand, \newenvironment and \renewenvironment. Starred versions and optional arguments are not handled. The following is quoted from this answer by Willie Wong to another question, and has more details about the limitations:

Based on the suggestion of Torbjørn T. and cfr, I looked more deeply into the de-macro package. It works to a certain extent. The following are caveats:

  • Unlike the what the documentation suggests, the version I have installed creates the database file <filename> instead of <filename>.db. However evidently it tests for <filename>.db as the name for the definitions database. So in its current incarnation it will recreate the definitions database from scratch every single run. For small documents this is not an issue. For larger documents one should copy (not move!) the database to <filename>.db to take advantage of any potential speed-up.

  • There are still some bugs to be ironed out. Occasionally it mangles preambles by inserting spurious } to the code. I have not yet found the reason why or the trigger/MWE for it. The small test cases I tried it on all worked fine in this regard.

  • Very important: as the documentation suggests, all the definitions you want swapped out must live inside a separate package ending in -private.sty. In the main .tex file one must use that package.

  • Also very important: the program handles \newcommand and \renewcommand, but not the starred variant \newcommand* (though this can be fixed by amending the regular expression a bit in the python code, I suppose). This is why my first attempt failed. (I always use the starred variant since I learned about it being best practice.)

  • Also very important: after removing the stars, the program threw up an error. Which I eventually figured out is due to my habit of writing \newcommand\cmdname{<replacement} instead \newcommand{\cmdname}{<replacement>}. That extra pair of braces is important for the parsing!

  • Lastly, very disappointing for me, the program cannot handle optional arguments. \newcommand{\cmdname}[2]{blah #1 blah #2} works fine, but \newcommand{\cmdname}[2][nothing]{blah #1 blah #2} throws an exception.

The problem with the star and the braces I can easily fix/workaround by rewriting my macro definitions (which as you remember, will be thrown away at the end anyway being the point of this exercise) without the stars and adding the extra braces.

The problem with optional argument handling however, currently makes the program somewhat less useful for me. I can work around it for now by splitting the optional and non-optional commands into two separate ones. Maybe, if I have some time in the future, I will look into adding support for it, after figuring out the logic of the original python script.

share|improve this answer
    
It seems to me that this would use an example showing its capabilities, in sense of optional arguments etc. –  yo' Jan 6 at 11:01
    
@tohecz Never used it myself, but I can take a look later. (If anyone else have used this, feel free to edit my answer.) –  Torbjørn T. Jan 6 at 11:49

Here's a perl script to do the same job. It has the same limitations as Paulo's code, but works well in your test case. I don't doubt it could be improved upon :)

You use it in the following way

perl replacenewcommands.plx myfile.tex

which outputs to the terminal, or

perl replacenewcommands.plx myfile.tex > outputfile.tex

which will output to outputfile.tex

replacenewcommands.plx

#!/usr/bin/perl

use strict;
use warnings;

# for newcommands
my @newcommandmacro=();
my %newcommandcontent=();

# for DeclareMathoperator
my @declaremathoperator=();
my %declaremathoperatorcontent=();

# for use as an index
my $macro;

# loop through the lines in the INPUT file
while(<>)
{
    # check for 
    #   \newcommand...
    # and make sure not to match
    #   %\newcommand
    # which is commented
    if($_ =~ m/\\newcommand{(.*)}{(.*)}/ and $_ !~ m/^%/)
    {
        push(@newcommandmacro,$1);
        $newcommandcontent{$1}=$2;

        # remove the \newcommand from the preamble
        s/\\newcommand.*//;
    }


    # loop through the newcommands in the 
    # main document
    foreach $macro (@newcommandmacro)
    {
      # make the substitution, making sure to escape the \
      # uinsg \Q and \E for begining and end respectively
      s/\Q$macro\E/$newcommandcontent{$macro}/g;
    }

    # check for 
    #   \DeclareMathOperator...
    # and make sure not to match
    #   %\DeclareMathOperator
    # which is commented
    if($_ =~ m/\\DeclareMathOperator{(.*)}{(.*)}/ and $_ !~ m/^%/)
    {
        push(@declaremathoperator,$1);
        $declaremathoperatorcontent{$1}=$2;

        # remove the \DeclareMathOperator from the preamble
        s/\\DeclareMathOperator.*//;
    }

    # loop through the DeclareMathOperators in the 
    # main document
    foreach $macro (@declaremathoperator)
    {
      # make the substitution, making sure to escape the \
      # uinsg \Q and \E for begining and end respectively
      s/\Q$macro\E(\(.*\))/\\operatorname{$declaremathoperatorcontent{$macro}}$1/g;
    }
    print $_;
}

In your test case

myfile.tex (original)

\documentclass{amsart} 
\usepackage{amsmath,amssymb}
\newcommand{\N}{\mathbb{N}}
\newcommand{\mycommand}{something else}
\DeclareMathOperator{\End}{End}

\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's $\N$ denote this ring by $\End(A)$. Throughout the lecture, $\N$ will denote
the set of natural numbers. \mycommand

and \mycommand again
\end{document} 

outputfile.tex (new)

\documentclass{amsart} 
\usepackage{amsmath,amssymb}




\begin{document}
In this lecture we'll study the ring of Endomorphisms of an Abelian group $A$.
Let's $\mathbb{N}$ denote this ring by $\operatorname{End}(A)$. Throughout the lecture, $\mathbb{N}$ will denote
the set of natural numbers. something else

and something else again
\end{document} 
share|improve this answer
    
Awesome, Chris! :) –  Paulo Cereda Aug 18 '12 at 19:58

This answer is a complement to Paulo Cereda's answer

As it stands, the regex he uses doesn't support nested brackets, and, indeed, the language doesn't allow for nested, recursive structures of arbitrary depth.

That said, we can robustify the regex for one additional level of nesting, like this:

r"\\newcommand{\\([A-Za-z]*)}{((:?[^{}]*{[^}]*})*[^}]*)}"
**             **              |****************|
#1             #2              #3

Notes:

  1. The Python r"..." construction for regexes is helpful because it doesn't need Python's text escaping (though it still requires regex escaping).
  2. That means we only need \ to represent \ in the regex, making us all less cross-eyed
  3. The main addition is a subpattern that looks for zero or more balanced brackets, followed by the tail end of the original pattern.

It's also possible to improve efficiency slightly by pre-compiling some of the regexes. Here's my adaptation of Paulo's solution:

#! python.exe

import re
import sys

if len(sys.argv) != 3:
    print('We need two arguments.')
    sys.exit()

mathDictionary = {}
commandDictionary = {}

inputHandler = open(sys.argv[1], 'r')
outputHandler = open(sys.argv[2],'w')

mathpat    = re.compile( r"\\DeclareMathOperator{\\([A-Za-z]*)}{((:?[^{}]*{[^}]*})*[^}]*)}" )
searchmath = mathpat.search 
newcpat    = re.compile( r"\\newcommand{\\([A-Za-z]*)}{((:?[^{}]*{[^}]*})*[^}]*)}" )
searchnewc = newcpat.search 

print('Processing...')

for line in inputHandler:
    mathOperator = searchmath(line)
    if mathOperator:
        mathDictionary[mathOperator.group(1)] = mathOperator.group(2)
    newCommand = searchnewc(line)
    if newCommand:
        commandDictionary[newCommand.group(1)] = newCommand.group(2)
    current = line
    for x in mathDictionary:
        current = re.sub(mathpat, '', current)
        current = re.sub(r"\\" + x + r"(?!\w)", '\\operatorname{' + mathDictionary[x] + '}', current)
    for x in commandDictionary:
        current = re.sub(newcpat, '', current)
        current = re.sub(r"\\" + x + "(?!\w)", commandDictionary[x], current)
    outputHandler.write(current)

inputHandler.close()
outputHandler.close()

print('Done.')

I have also made it one-pass; defining a newcommand after using it doesn't make too much sense.

share|improve this answer
    
By the way, if you think from this that regex is for loonies, you're probably right. –  Brent.Longborough Jan 31 at 23:21

I wrote some javascript for expanding macros defined by \def, \gdef, \edef, \xdef, \newcommand, \newcommand*, \renewcommand, \renewcommand*, \DeclareMathOperator and \DeclareMathOperator*. You may try it here.

function expandMacros(tex) {
    function nestBrackets(level) {
        var level = level || 5, re = c = "(?:[^\\r\\n\\{\\}]|\\\\[\\{\\}]|\\r?\\n(?!\\r?\\n))*?";
        while (level--) re = c + "(?:\\{" + re + "\}" + c + ")*?";
        return " *(\\{" + re + "\\}|[^\\{])";
    }    
    function getRegExp(name, macro) {
        var num = macro.num, def = macro.def, re = "";
        while (num--) re += nestBrackets();
        re = "\\" + name + "(?![a-zA-Z\\}])" + re;
        return new RegExp(re, "g");
    }
    function trimString(s) {
        return s.replace(/^ +| +$/g, '').replace(/^\{|\}$/g, "");
    }
    function extractMacros() {
        var cs = "\\\\\\w+", re;
        // \def, \gdef, \edef and \xdef
        re = new RegExp("\\\\[gex]?def\\*? *(" + cs + ") *(#\\d)*" + nestBrackets(), "g");
        tex = tex.replace(re, function(match){
            var m = arguments;
            var macro = {
                num:  m[2] ? Math.min(m[2].length / 2, 9) : 0,
                def:  trimString(m[3])
            };
            macros[trimString(m[1])] = macro;
            return "";
        });
        // \newcommand, \newcommand*, \renewcommand and \renewcommand*
        re = new RegExp("\\\\(?:re)?newcommand\\*? *(" + cs + "|\\{" + cs + "\}) *(\\[(\\d)\\])?"
                        + nestBrackets(), "g");
        tex = tex.replace(re, function(match){
            var m = arguments;
            var macro = {
                num:  m[3] || 0,
                def:  trimString(m[4])
            };
            macros[trimString(m[1])] = macro;
            return "";
        });
        // \DeclareMathOperator and \DeclareMathOperator* inside amsmath
        re = new RegExp("\\\\DeclareMathOperator(\\*?) *(" + cs + "|\\{" + cs + "\}) *"
                        + nestBrackets(), "g");
        tex = tex.replace(re, function(match){
            var m = arguments;
            var macro = {
                num:  0,
                def:  "\\operatorname" + m[1] + "{" + trimString(m[3]) + "}"
            };
            macros[trimString(m[2])] = macro;
            return "";
        });
    }
    function replaceMacros() {
        var i = 0, m, re, num;
        for (name in macros) {
            m = macros[name];
            re = getRegExp(name, m), num = m.num;
            //console.log(re);
            tex = tex.replace(re, function(match){
                //console.log(arguments);
                var args = [], result = m.def, k;
                for (k = 1; k <= num; k++) {
                    args[k] = trimString(arguments[k]);
                }
                //console.log(args);
                for (k = 1; k <= num; k++) {
                    result = result.replace(new RegExp("#" + k, "g"), args[k]);
                }
                return result;
            });
        }
    }
    var macros = {};
    extractMacros();
    replaceMacros();
    return tex;
}

document.getElementById("run").onclick = function() {
    var input = document.getElementById("input"),
        output = document.getElementById("output");
    output.value = expandMacros(input.value);
}
share|improve this answer
    
Oh, Thank you very much for something that works online! Will the link always work? (The thing about codes like this is that, they would help reduce manual labour. So, I am pretty happy with this implementation!) –  kan Jan 13 at 13:38
    
In fact, the web page is hosted by http://jsfiddle.net. Therefore the link will always work as long as JSFiddle website is online. You may also save the web page to your computer. –  Z.H. Jan 13 at 14:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.