# How to extract parts of (La)TeX document?

I want to run queries against LaTeX documents like these:

1. get list of all equations, even if one declares \def\be{} and uses it;
2. get list of all arguments of some command (e. g. \marginpar), where it has been used in document (again, even if one declares \def\m\marginpar and calls this macro);
3. get the theorem with given counter value.

The task seems to require full compiling of all macros, but I want to get arguments in not expanded form --- that is not as blocks and glue, but as user's input (if possible).

How can I do such things?

Update: ideally, I want to use "semantic markup" to mark parts of documents and generate different views of them. Assume I have smth like this:

\beginBlock{calculus}
\beginBlock{content}
... lectures...
\endBlock
\beginBlock{exam}
... problems...
\endBlock
\endBlock[exam]
\endBlock[calculus]


and similar for linear algebra in other file; I want to generate a report with sections for calculus and linear algebra, containing only exam problems, e. g.

-

## migrated from stackoverflow.comApr 8 '12 at 3:07

This question came from our site for professional and enthusiast programmers.

This has to be done by overloading the respective constructs, one by one. LaTeX does not provide a general interface for this kind of thing. It might just be that LaTeX3 does, but I don't know. If you post a minimal example demonstrating what you wish to achieve in particular, people can give hints how to acieve this. – Stephan Lehmke Apr 8 '12 at 3:14
@Stephan-Lehmke Thank you, I've added example. – Alexei Golovko Apr 9 '12 at 18:55

Let's look at \marginpar. You want to redefine this command to save its arguments in some list to be printed, say, at end document.

One has to look the definition of \marginpar, discovering that it's not a command with arguments and it performs a rather complicated series of tasks. So we can't simply add some action to it without examining in detail the macro that are used.

A good strategy is to gather the arguments and then put them back for the original \marginpar.

\makeatletter
\let\saved@marginpar\marginpar
\renewcommand\marginpar[2][]{%
\if\relax\detokenize{#1}\relax
\let\saved@mp@optional\@empty
\else
\def\saved@mp@optional{[#1]}%
\fi
\def\saved@mp@mandatory{{#2}}%
\expandafter\expandafter\expandafter\saved@marginpar
\expandafter\saved@mp@optional\saved@mp@mandatory}
\newtoks\saved@mp@toks
\def\printmarginpars{\the\saved@mp@toks}
\def\saved@mp@print#1#2{\par\noindent Marginpar\\
Optional: #1\\
Mandatory: #2\par\medskip}
\makeatother


If you say, after this code,

\newcommand{\mpx}{\marginpar}


then it will work the same.

What do the macro do?

1. \marginpar is redefined to have an optional and a mandatory argument, after having saved a copy of the original command.

2. If the optional argument is empty, then we define \saved@mp@optional to be \@empty, otherwise we define it to expand to the optional argument in brackets.

3. Similarly, we define \saved@mp@mandatory to expand to the mandatory argument in braces.

4. We add the contents of the arguments to the token list we have allocated.

5. We put back the arguments after \saved@marginpar, so LaTeX will do its usual work as if nothing had happened.

## Example

\documentclass{article}
\makeatletter
\let\saved@marginpar\marginpar
\renewcommand\marginpar[2][]{%
\if\relax\detokenize{#1}\relax
\let\saved@mp@optional\@empty
\else
\def\saved@mp@optional{[#1]}%
\fi
\def\saved@mp@mandatory{{#2}}%
\expandafter\expandafter\expandafter\saved@marginpar
\expandafter\saved@mp@optional\saved@mp@mandatory}
\newtoks\saved@mp@toks
\def\printmarginpars{\the\saved@mp@toks}
\def\saved@mp@print#1#2{\par\noindent Marginpar\\
Optional: #1\\
Mandatory: #2\par\medskip}
\makeatother

\newcommand{\mpx}{\marginpar}

\begin{document}
a\marginpar{xxx}
b\mpx[yyy]{zzz}

\printmarginpars
\end{document}

-
Great thanks. If I want print to another file "sources" of marginpars, how can I get corresponding non-expanded string? (that is from \mpx{\z} I want extract string \z, not its expansion.) I have tried Mandatory: \expandafter{#2} and similar, but it does not work. – Alexei Golovko Apr 9 '12 at 18:21
@AlexeiGolovko Mandatory: \texttt{\detokenize{#2}} (the \texttt is just to pretty print the result). – egreg Apr 9 '12 at 19:20
Thank you again. And is it possible to use similar trick when a text which should be extracted is not in one group {...} but is delemited by commands like $$...$$? – Alexei Golovko Apr 10 '12 at 7:20
@AlexeiGolovko Yes: check the environ package. However, doing it on a command and environment basis requires redefining each of them. Your edit doesn't explain very clearly what you have in mind. – egreg Apr 10 '12 at 8:51
I have in mind something like tagging package, with ability include content of taggedblock which is nested in excluded taggedblock (so that the content in outermost block, which is not in the inner block, should not be presented). Hope, such description is more clear. – Alexei Golovko Apr 10 '12 at 16:04

I think you will be better off with an external script in your favourite language such as python or perl.

If you want to do it with LaTeX an approach that could work, at least for the command that are not environments is to save the contents of the commands either on file or within a list.

I will demonstrate it for the use of marginpars. You first define a list:

  \def\alist{}


and then you define macros to add to the list:

  \def\addtolist#1#2{
}

\long\expandafter\gdef\csname#1\endcsname{\textbf{#1}: #2}
}


Notice on the last one, you save the contents of the marginpar by using addmp and then you print it.

Full MWE:

\documentclass[11pt]{book}
\usepackage{lstdoc,lipsum}
\begin{document}
\makeatletter
\def\alist{}

}

\long\expandafter\gdef\csname#1\endcsname{\textbf{#1}: #2}
}

\def\PrintMP{%
\@for \i:=\alist\do{%
\csname\i\endcsname}
}
%example
% print the marginpars
\Printmp
\makeatother
\end{document}


Edit

Based on new information provided by the OP, one could use a .dtx file with the doc/docstrip system. This would allow you to save into different files sections of the document etc.

-
For equations it should also not be too hard when using amsmath, as amsmath collects the environment content for measuring, hence it could be saved at this point. – Stephan Lehmke Apr 8 '12 at 10:27
@StephanLehmke Not for equation or equation* – egreg Apr 8 '12 at 13:30
@egreg You're right. I was remembering that amsmath redefines equation, but apparently it does so without collecting the environment body. I wonder however what harm would be done by simply redefining equation in this sense. – Stephan Lehmke Apr 8 '12 at 14:40
@StephanLehmke It can be done, of course. – egreg Apr 8 '12 at 14:43
Why using \lst@lAddTo which requires listings when there's already \g@addto@macro? – egreg Apr 8 '12 at 14:44

Ok, now I understand this better. It is a rather common idea to have one "master" document which will produce different output in different "modes".

This would mean you'd make different "configurations" where the environments you mention are defined differently. By activating one "configuration" with a specific definition of the environments you get a different version of the document. One "configuration" could be a .tex file containing environment definitions (one of which is then \input) or different options for a dedicated package which represent different environment definitions.

Packages which might be relevant: comment, optional, versions, version, renditions, tagging.

Another very sophisticated, large package which can be used for this kind of thing: The AcroTeX eDucation Bundle.

Questions which might be relevant:

Showing Solutions of the Questions “separately”

Comparing packages which facilitate typesetting exercises and solutions: exercise vs. answers vs. probsoln

Commenting out large sections

Including only the parts of a document corresponding to a conditional expression

There was another question highly relevant to this, but I fail to find it now. It was concerned with how to store different content parts in a common file, assembling different documents from them. Maybe someone else remembers it.

-
Thank, tagging looks especially nice. There is still remaining question, how include content of block, nested in excluded block. Maybe, I can combine tagging with egreg's approach from the first answer above; I shall think and try. – Alexei Golovko Apr 10 '12 at 15:49
This can get really ugly in TeX. A huge amount of work if it doesn't already exist in some package. I'd consider storing the whole thing as XML (only the structure which is build by \beginBlock...\endBlock really needs to be modeled as XML tags) and implementing the ex-/inclusion with XSL. The result could easily be a .tex` file again, no need to parse XML with TeX. Or you could input the whole XML with LuaTeX and do the parsing/ex-/including with a Lua XML library. – Stephan Lehmke Apr 11 '12 at 0:59