# How to randomly extract some pages of a PDF file, sort their pages, and finally bundle them into a new PDF?

Some books in amazon.com have excerpts (first several pages plus appendixes) from which we can know their contents, layouts, etc roughly prior to purchasing. Statistically I think it will be better if an excerpt is made by extracting some pages (30-50 pages might be enough) of the corresponding book at random, sort their pages in increasing order, and finally bundle them in to a new PDF.

My question is: how to do it in LaTeX?

## Minimal Working Example

% compile with pdflatex -shell-escape
% =============================================================================================
\def\NoticeThatIAmUsingThisPackageToExtractSomePagesFromAnExternalPDFFileInMyComputer{pdfpages}
% =============================================================================================
\documentclass{article}
\usepackage{filecontents}
\begin{filecontents*}{book.tex}
\documentclass{book}
\usepackage{blindtext}
\begin{document}
\Blinddocument
\end{document}
\end{filecontents*}

\immediate\write18{pdflatex book.tex}
\immediate\write18{pdflatex book.tex}

\usepackage{\NoticeThatIAmUsingThisPackageToExtractSomePagesFromAnExternalPDFFileInMyComputer}

\def\NumberOfPagesOfExcerpt{50}

\begin{document}

% do randomization, sorting and bundling here!
% \includepdf[pages=-]{book}
\end{document}

-
It seems like pdfpages would be the way to go here. –  Matthew Leingang Apr 2 '13 at 4:05
You could use pdftk pdflabs.com/tools/pdftk-the-pdf-toolkit –  s0rce Apr 2 '13 at 4:31
Karl'sstudents: I think what @MatthewLeingang is saying that with the pdfpages package you can use something like \includepdf[pages={3,12,19,33,5}]{filename.pdf} etc to insert the desired pages from filename.pdf. So all you need is to randomly generate the page numbers and there are several solution on this site for generating random numbers. –  Peter Grill Apr 2 '13 at 5:03
@PeterGrill: That should be obvious if he parsed my MWE. :-) –  kiss my armpit Apr 2 '13 at 5:08
@Karl'sstudents: You should have made this more clear in the question then, not hiding it inside code. –  Martin Scharrer Apr 2 '13 at 9:34

A lualatex solution:

\documentclass{article}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents*}{book.tex}
\documentclass{book}
\usepackage{blindtext}
\begin{document}
\Blinddocument
\end{document}
\end{filecontents*}

\begin{luacode}
function get_random_pages(randPages,totalPages, randSeed)
--[[--
Constructs a sorted list of randPages random page numbers within a range 1..totalPages
@Parameter: randPages
The number of random pages to extract
@Parameter: totalPages
Total number of pages in a pdf file
@Parameter: randSeed
Random seed
--]]--
local pagesLeft= {}
local pageList = {}
for pageNo=1, totalPages, 1 do
table.insert(pagesLeft,pageNo)
end

math.randomseed (randSeed)

local r
for i=1, randPages do
r=math.random(#pagesLeft)
table.insert(pageList,pagesLeft[r])
table.remove(pagesLeft,r)
end
table.sort(pageList)
local s="\\includepdf[pages={"
s=s..pageList[1]
for i=2, randPages do
s=s..","..pageList[i]
end
s=s.."}]{book}"
tex.print(s)
end
\end{luacode}

\immediate\write18{pdflatex book.tex}
\immediate\write18{pdflatex book.tex}

\usepackage{pdfpages}

\def\NumberOfPagesOfExcerpt{9}
\def\NumberOfPagesInPdf{17}
\def\randomSeed{27449}

\begin{document}

% do randomization, sorting and bundling here!
\directlua{get_random_pages(\NumberOfPagesOfExcerpt,\NumberOfPagesInPdf,\randomSeed)}

\end{document}


Process with lualatex -shell-escape random_pages.tex.

Edit:

• a standard table.concat function used as suggested by @Aditya,

• a command \randomPages defined with an optional random seed parameter,

• the number of pages in pdf is defined by means of pdftex primitives as in here.

random_pages.tex:

\documentclass{article}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents*}{book.tex}
\documentclass{article}
\usepackage{geometry}
\geometry{
paperwidth=74mm,
paperheight=105mm,
margin=2em,
bottom=9ex,
}
\usepackage{blindtext}
\begin{document}
\Blinddocument
\end{document}
\end{filecontents*}

\begin{luacode}
function get_random_pages(randPages,totalPages, randSeed)
--[[--
Constructs a sorted list of randPages random page numbers within a range 1..totalPages
@Parameter: randPages
The number of random pages to extract
@Parameter: totalPages
Total number of pages in a pdf file
@Parameter: randSeed
Random seed: used only if >0
--]]--
local pagesLeft= {}
local pageList = {}
for pageNo=1, totalPages, 1 do
table.insert(pagesLeft,pageNo)
end
if randSeed>0 then math.randomseed(randSeed) end
local r
for i=1, math.min(randPages,totalPages) do
r=math.random(#pagesLeft)
table.insert(pageList,pagesLeft[r])
table.remove(pagesLeft,r)
end
table.sort(pageList)
local s="\\includepdf[pages={"
s=s..table.concat(pageList,",")
s=s.."}]{book}"
tex.print(s)
end
\end{luacode}

\immediate\write18{pdflatex book.tex}
\immediate\write18{pdflatex book.tex}

\usepackage{pdfpages}

\def\NumberOfPagesOfExcerpt{42}
\def\randomSeed{27449}

\makeatletter
\newcommand\@randomPages[3]{%
\pdfximage{#2}%
\def\NumberOfPagesInPdf{\the\pdflastximagepages}%
\directlua{get_random_pages(#1,\NumberOfPagesInPdf,#3)}%
}
\def\randomPages{%
\@ifnextchar[{\@with}{\@without}}%
\def\@with[#1]#2#3{%
\@randomPages{#2}{#3}{#1}%
}%
\def\@without#1#2{%
\@randomPages{#1}{#2}{0}%
}%
\makeatother

\begin{document}
% do randomization, sorting and bundling here!

%  \randomPages[\randomSeed]{10}{book.pdf} % supposed to produce a fixed set of pages every time
\randomPages{10}{book.pdf}         % supposed to produce a different set of pages every time

\end{document}

-
You can use table.concat function to concat the table to a string. –  Aditya Apr 2 '13 at 7:55
@Aditya: You mean table.concat(pageList,",") ? Yes, thanks for the tip. The code just uses basic commands to make it easier to recode in some other language. –  g.kov Apr 2 '13 at 8:13

Here is a solution that written in ConTeXt Lua Documents. Modify the parameter filename and n appropriately (I'll post a version that uses command line parameters later).

Save this as filter.cld (note the extension!), and process using context filter.cld.

local random = math.random
local format = string.format

-- Sample n items out of m without replacement
function reservoirsample (n, m)
local sampledlist = {}
if n == 0 then return sampledlist end
for i = 1, m do
-- Take the first n samples
if i <= n then
sampledlist[i] = i
else
-- Randomly replace one sample
local j = random(i)
if j < n then
sampledlist[j] = i
end
end
end
table.sort(sampledlist)
return sampledlist
end

local filename="fonts-mkiv.pdf"
local n = 20

context.starttext()

-- Example taken from grph-inc.lua
local fig = figures.push { name = filename }
figures.identify()
figures.check()
local nofpages = fig.used.pages
figures.pop()

selected = reservoirsample(n, nofpages)

print(format("::: File %s has %d pages, selecting %d", filename, nofpages, n))
print(format("::: %s", table.concat(selected, ", ")))

for i = 1,#selected do
context.startTEXpage()
context.externalfigure( {filename}, {page=selected[i]} )
context.stopTEXpage()
end

context.stoptext()

-

Here is a solution that only uses the mathy and loopy bits of pgf. It borrows some code Mark Wibrow wrote on the pgf-users mailing list a while back for shuffling a pgfmath list. Lists in pgfmath are implemented with a hash, rather than a single token list.

To get a random list of k elements from the list {1,...,N}, I create the list {1,...,N} and knuth shuffle it. Then I bubble-sort the first k elements. Then for each i from 1 to k I include the i-th page of the PDF.

\documentclass{article}
\usepackage{pgf,pgffor}
\usepackage{pdfpages}

\makeatletter

% declare a list by its elements
% e.g., \pgfmathdeclarelist{mylist}{{foo}{bar}{baz}}
\def\pgfmathdeclarelist#1#2{%
\def\pgfmath@list@name{#1}%
\c@pgfmath@counta=0%
\pgfmath@declarelistlist#2{\pgfmath@stop}%
}%
\def\pgfmath@declarelistlist#1{%
\ifx#1\pgfmath@stop%
\expandafter\edef\csname pgfmath@list@\pgfmath@list@name @length\endcsname{\the\c@pgfmath@counta}%
\else%
\pgfutil@namedef{pgfmath@list@\pgfmath@list@name @\the\c@pgfmath@counta}{#1}%
\expandafter\pgfmath@declarelistlist%
\fi%
}

% get a list item
% \pgfmathgetlistitem{\cs}{mylist}{3} lets \cs be the 3rd item of mylist
\def\pgfmathgetlistitem#1#2#3{%
\expandafter\let\expandafter#1\expandafter=\csname pgfmath@list@#2@#3\endcsname%
}

% set a list item
% \pgfmathsetlistitem{mylist}{3}{foo} defines the 3rd item of mylist to be foo
% caution - you may need the 3rd argument expanded first.
\def\pgfmathsetlistitem#1#2#3{%
\pgfutil@namedef{pgfmath@list@#1@#2}{#3}%
}

% get the length of a list
% \pgfmathgetlistlength{\mylistlength}{mylist} lets \mylistlength be the length of the list.
\def\pgfmathgetlistlength#1#2{%
\expandafter\let\expandafter#1\expandafter=\csname pgfmath@list@#2@length\endcsname%
}

% set the length of a list
% \pgfmathsetlistlength{mylist}{length} defines the length of mylist to be length
\def\pgfmathsetlistlength#1#2{%
\expandafter\edef\csname pgfmath@list@#1@length\endcsname{#2}
}

\def\pgfmathknuthshuffle#1{%
\pgfmathgetlistlength\pgfmath@len{#1}%
\pgfmathloop%
\ifnum\pgfmathcounter>\pgfmath@len%
\else%
\pgfmathrandominteger\pgfmath@temp{1}{\pgfmath@len}%
\pgfmathgetlistitem\pgfmath@@temp{#1}{\pgfmathcounter}%
\pgfmathgetlistitem\pgfmath@@@temp{#1}{\pgfmath@temp}%
\def\pgfmath@marshal{\pgfmathsetlistitem{#1}}%
\expandafter\pgfmath@marshal\expandafter{\expandafter\pgfmath@temp\expandafter}\expandafter{\pgfmath@@temp}%
\expandafter\pgfmath@marshal\expandafter{\expandafter\pgfmathcounter\expandafter}\expandafter{\pgfmath@@@temp}%
\repeatpgfmathloop%
}

\def\NumberOfPagesOfExcerpt{9}
\def\NumberOfPagesInPdf{17}

% Populate page list. Rather than use \pgfmathdeclarelist we allocate the list and assign in a loop.
% sorry for the \global... pgf's \foreach creates a group.
\def\s@pagelist{pagelist} % makes expansion easier
\pgfmathsetlistlength{pagelist}{\NumberOfPagesInPdf}
\foreach \i in {1,...,\NumberOfPagesInPdf}{
\global\expandafter\pgfmathsetlistitem\expandafter\s@pagelist\expandafter\i\expandafter{\i}
}

\pgfmathknuthshuffle{pagelist}

% now a bubble sort on the first \NumberOfPagesOfExcerpt items in the list.
\pgfmathtruncatemacro{\n}{\NumberOfPagesOfExcerpt-1}
\foreach \j in {1,...,\n}{
\pgfmathtruncatemacro{\k}{\NumberOfPagesOfExcerpt-\j}
\foreach \i in {1,...,\k}{
\pgfmathtruncatemacro{\iplusone}{\i+1}
\pgfmathgetlistitem{\pagei}{pagelist}{\i}
\pgfmathgetlistitem{\pageiplusone}{pagelist}{\iplusone}
\ifnum\pagei>\pageiplusone
\global\expandafter\pgfmathsetlistitem\expandafter\s@pagelist\expandafter\i\expandafter{\pageiplusone}
\global\expandafter\pgfmathsetlistitem\expandafter\s@pagelist\expandafter\iplusone\expandafter{\pagei}
\fi
}
}

\makeatother

\begin{document}

\foreach \i in {1,...,\NumberOfPagesOfExcerpt}{
\pgfmathgetlistitem{\pagei}{pagelist}{\i}
\includepdf[pages=\pagei]{book.pdf}
}

\end{document}


As you can see, it's a bit messy, but it doesn't require lua or an external script. IANACS so I don't know how efficient it is, either. But if you wanted efficiency you wouldn't do this job in TeX. :-)

-

Calling an external randomizer named excerpting.exe from within LaTeX.

## LaTeX Code:

% compile with pdflatex -shell-escape
\documentclass{article}
\usepackage{pdfpages}
\def\bookfilename{status-lua}% http://chat.stackexchange.com/transcript/41?m=8712421#8712421
\def\take{30}
\def\seeder{1}
\def\auxiliaryfilename{random.txt}

\pdfximage{\bookfilename.pdf}
\immediate\write18{excerpting \the\pdflastximagepages\space \take\space \seeder\space \auxiliaryfilename}

\begin{document}
\loop
\includepdf[pages=\data]{\bookfilename}
\repeat
\end{document}


## C# (Fisher-Yates Shuffle):

// excerpting.cs
using System;
using System.IO;
using System.Linq;

namespace Excerpting
{
class Program
{
static void Main(string[] args)
{
int total = int.Parse(args[0]);
int take = int.Parse(args[1]);
int seeder = int.Parse(args[2]);
string filename = args[3];

int[] array = Enumerable.Range(1, total).ToArray();

Random random = new Random(seeder);
for (int i = total - 1; i > 0; i--)
{
int j = random.Next(i+1);
int temp = array[i];
array[i] = array[j];
array[j] = temp;
}

File.WriteAllLines(filename, array.Take(take).OrderBy(x => x).Select(x => x.ToString()));
}
}
}


## C# (Random Sort):

Someone claims that Random Sort has uniform distribution but I have not checked it yet.

// excerpting.cs
using System;
using System.IO;
using System.Linq;

namespace Excerpting
{
class Program
{
static void Main(string[] args)
{
int total = int.Parse(args[0]);
int take = int.Parse(args[1]);
int seeder = int.Parse(args[2]);
string filename = args[3];

Random random = new Random(seeder);
string[] array = Enumerable.Range(1, total)
.OrderBy(x => random.Next())
.Take(take)
.OrderBy(x => x)
.Select(x => x.ToString())
.ToArray();

File.WriteAllLines(filename, array);
}
}
}

-