Invisible markers in PDFs using pdflatex

Question

I'm using latexdiff to show changes that I make in a large text (200+ pages). Most of my changes affect only a few pages though, so I don't want my readers to sift through 200 pages looking for the rare pages where isolated typo corrections were made: only about 30pp out of the 200 will contain changes.

I would like to:

1. Insert hidden PDF markers on pages where a changes occur
2. Compile with pdflatex

3. Run a script that filters out pages that don't contain the hidden PDF markers

   book.tex -latexdiff-> book_diff.tex -pdftex-> 200pp.pdf -script-> 30pp.pdf

Could someone recommend:

• a tex package for inserting "markers" of some sort in a PDF file
• a python package for reading PDFs (and looking for said markers)

Answer 1

My first thought was to use the I/O file operations, e.g. \write or \immediate\write, but it faced a known problem with floating objects and their behavior. I had also problem around floating objects with own counter when there was no \global prefix.

I rather used cross-references directly where a protected write is ensured. I can offer you this solution with the help of LuaTeX. The processed steps are:

• We typeset a regular document and mark a position. For purpose of this post it is a visible mark and it is a page number. The definition uses regular \label command with a certain prefix, e.g. malipivo. I named the file filter.tex and the command \writeme. The command uses own counter and that assures the uniqueness of the marker in the \label command.
• The filter.tex file is processed several times depending on an actual typeset material, in this example it is two times. After the first run we are getting a document with two red question marks (row 1 in the preview), after the second run we are getting cross-references correctly typeset (row 2 in the preview).

This is the code:

%! *latex filter.tex
%! Run me twice or even more times to get references right.
\def\myprefixis{malipivo}

\documentclass[a4paper]{article}
\usepackage{tikz}
\usepackage[numbers]{kantlipsum}
\newcount\mycounter 
\mycounter=0
% Our own marker...
% Save the reference and mark it...
\def\writeme{%
\global\advance\mycounter by 1%
\label{\myprefixis\the\mycounter}%
\begin{tikzpicture}%
\begin{pgfinterruptboundingbox}%
\node[red,font=\bfseries\Huge,scale=4]{\pageref{\myprefixis\the\mycounter}};%
\end{pgfinterruptboundingbox}%
\end{tikzpicture}%
  }% End of \writeme...

\begin{document}
\section{My experiment}
\kant[1-2]\writeme
\kant[14]\writeme
\kant[15-16]\writeme
\kant[25]\writeme\ There is a typo and\writeme\ one more\writeme
\kant[26-32]\writeme
\kant[46]\writeme
\begin{figure}[p]
\kant[51] I am floating.\writeme
\end{figure}
\kant[52-61]\writeme
\end{document}

The content of the filter.aux file looks like this:

\relax 
\@writefile{toc}{\contentsline {section}{\numberline {1}My experiment}{1}}
\newlabel{malipivo1}{{1}{1}}
\newlabel{malipivo2}{{1}{1}}
\newlabel{malipivo3}{{1}{2}}
\newlabel{malipivo4}{{1}{2}}
\newlabel{malipivo5}{{1}{2}}
\newlabel{malipivo6}{{1}{2}}
\newlabel{malipivo7}{{1}{4}}
\newlabel{malipivo8}{{1}{4}}
\newlabel{malipivo10}{{1}{7}}
\newlabel{malipivo9}{{1}{8}}

You can actually see, that there is some floating object as numbers are swapped to their page number counterparts (malipivo10<->malipivo9; 7<->8). We need to process this auxiliary file somehow and extract the pages from filter.pdf. I used LuaTeX and the pdfpages package.

• At first I thought that I needed to sort the lines, but it seems it is not necessary, it is already sorted by the page numbers, not by the markers.

• However, I needed to extract lines with malipivo prefix included and skip the others. I used string.find from LuaTeX. There is a note in the terminal from this step saying Testing note on page 1....

• Then I needed to extract page numbers to get 1 1 2 2 2 2 4 4 7 8, I used the string.gsubfunction.

• I had to assure that each page will be included only once, so we should get this sequence of numbers 1 2 4 7 8 after removing duplicates.

• My next step was to add commas, I did that while concatenating the temporary string. We are getting to 1,2,4,7,8.

• The rest was relatively easy. I made a LaTeX command out of it to get \includepdf[pages={1,2,4,7,8},fitpaper]{filter.pdf} and shipped it to the document body.

I created a simple function in LuaTeX named testme with two parameters, the first parameter is asking for the tested file (filter), the code uses its aux and pdf files, and the second parameter is asking for a prefix we were using (in theory, there could be more prefixes), I used malipivo. The actual typesetting is done by \directlua{testme("filter","malipivo")}.

I enclose the code (filtered.tex) to be run via lualatex once and finally a preview of selected pages (filtered.pdf) which were marked in the original code (filter.tex).

%! lualatex filtered.tex
%! run me only once
\documentclass{article}
\usepackage{pdfpages}
\usepackage{luacode}

\begin{document}

\begin{luacode*}
function testme (filtering, thekey)
-- Initializing variables...
local myinput=filtering..".aux" -- Where are the reference?
local myinputpdf=filtering..".pdf" -- Where are the real PDF pages?
local mylines="" -- List of pages to get is?
local mycount=0 -- How many tested reference do we have?
local lastcount="" -- What is the previous reference?
-- The core of the function...
for mylinetemp in io.lines(myinput) do -- Read input file line by line
  test=string.find(mylinetemp,thekey) -- Is a unique key involved? 
  if test~=nil then -- Yes, the key is present there...
    myline=string.gsub(mylinetemp,"\\newlabel{"..thekey..".-}{{.-}{(.-)}}", "%1", 1) -- Find a group with page reference.
    print("Testing note on page "..myline.."...") -- Print information to the terminal
    mycount=mycount+1 -- Increase a number of tested references
    if mycount==1 then -- Save the first value as the starting value
      mylines=myline -- Start the resulting string
      lastcount=myline -- Remember the last occurance
    else -- Add value to the list if not already stored in the list
      if lastcount~=myline then -- But only if last two values differ (aka uniq)
        mylines=mylines..","..myline -- Add comma and value to the string (aka join)
        lastcount=myline -- Remember this occurance as the last one for next testing
      end -- of if not lastcount
    end -- of if mycount
  end -- of if thekey is involved
end -- of myline
-- Print the results and generate PDF via regular typesetting...
local keyresult="\\includepdf[pages={"..mylines.."},fitpaper]{"..myinputpdf.."}"
print(keyresult)
tex.print(keyresult)
end -- of function testme
\end{luacode*}

% The tested file, e.g. filter.{aux,pdf}, and its reference label prefix.
\directlua{testme("filter","malipivo")}

\end{document}

Answer 2

Okay so I wrote a first draft of a working pipeline.

• As per @ChristianH's suggestion, I used pdfcomment
• For the script, I used:
  • pdfminer to read the PDF annotations
  • pdfrw to write the selected pages

The code is here: https://gist.github.com/ivanistheone/219dad11a30efef82b7e