PDF page references in TeX4ht generated html?

Question

I'm using TeX4ht to make html and epub versions of documents that are also processed to pdf. I'm unclear on how to include some kind of visual page references in the html and epub that correlate to the page numbering of the pdf ("master") document. As it is, there is no way for viewer to know what pdf page corresponds to his/her location in the html text.

There are a couple of internal page references (i.e, \pageref{xxx}) in the LaTeX/pdf versions that do get through to the html/epub versions, e.g., as "see page 46". When clicked on in html the link does indeed go to the area in the document that was page 46 in the pdf version (expected result I guess because link is actually to an embedded anchor and not to a specific page), but there is no data in the html file to indicate where the page breaks are in the LaTeX/pdf versions.

Is there some option in TeX4ht that I can set to have it automatically include page number codes in the html (e.g., "[*46]"). Or is there command I can include in the LaTeX file itself so that it inserts page markers that will be included in the html I get from TeX4ht?

Answer 1

I think that page numbers are important even in ebooks. In many books, especially in humanities, philosophy, etc., there are chapters without sections, long about 40, 50 pages. Page numbers are really necessary in such cases.

Other example where page numbers of the book's original layout are important, are braille books, where original page numbers are used together with page numbers of the braille document.

tex4ht doesn't have access to page numbers of the original document, as it uses special format of dvi file, where pages doesn't correspond to the pdf layout.

Using some preprocessing tool is possible, it is not so hard to extract for example last line of each page from the pdf file, but it would be hard to find that text in the tex file, as it can be generated from macros.

So I think only way is to use luatex, where we have access to the page contents after the page was assembled, or to the paragraphs before they are break to the lines. Downside is, that tex4ht doesn't yet have good support for the luatex, but I hope that in near future it will be better. See this answer of mine.

At the moment, I have working instance of the htlualatex (htlatex for luatex) at my linux destop at work, but not on my windows desktop at home, so I cannot test html generation of the code, only using distinct sizes of the pdf.

My idea is to compute checksums of all glyphs in the pdf file and save their value at every page break. Then, when we want to insert page break marks in file with different layout, load saved checksums, compute checksums of the new file and where they match, insert the mark. I know that it is not robust approach and thinks like figures, equations, etc., can broke it, so if anybody knows about better algorithm, I will be happy.

Now there is a sample:

\documentclass{book}

\usepackage{lipsum}
\usepackage[a4paper]{geometry}
\usepackage[]{milestone}
\begin{document}
\title{Milestone sample}
\author{Michal}
\maketitle
\chapter{First}
\section{Section}
\lipsum[2-12]
\subsection{Subsection}
\lipsum[2]
\chapter{Next chapter}
\lipsum[3-20]
\end{document}

Now we can set different page layout and insert the page marks with \usepackage[insert]{milestone}:

\documentclass{book}

\usepackage{lipsum}
\usepackage[a5paper]{geometry}
\usepackage[insert]{milestone}
\begin{document}
\title{Milestone sample}
\author{Michal}
\maketitle
\chapter{First}
\section{Section}
\lipsum[2-12]
\subsection{Subsection}
\lipsum[2]
\chapter{Next chapter}
\lipsum[3-20]
\end{document}

This is the file milestone.sty:

\ProvidesPackage{milestone}
\RequirePackage{luacode,luatexbase,etoolbox,kvoptions}
\DeclareBoolOption{insert}
\ProcessKeyvalOptions*
\begin{luacode}
milestone=require("milestone")
\end{luacode}
\ifmilestone@insert
\begin{luacode}
luatexbase.add_to_callback("pre_linebreak_filter",milestone.insertPageBreaks,"Insert")
\end{luacode}
\else
\AfterEndDocument{%
\directlua{milestone.writeMilestones()}
}

\newtoks\oldoutput
\oldoutput=\expandafter{\the\output}
\output{
    \directlua{
    milestone.pagenumber = "\thepage"
    milestone.getPageBreaks(tex.box[255].list)
    }
    \the\oldoutput
}
\fi

And the lua module milestone.lua:

--local node,texio,unicode,io,tex,dofile=node,texio,unicode,io,tex,dofile 
module("milestone",package.seeall)
-- Variables accessible from the outside
milestonefile = tex.jobname..".mil"   -- File, where all milestones are saved
pagenumber=""                         -- Pagenumbers must be saved in TeX, 
                                      -- in the output routine

local glyph = node.id('glyph')
local milchecksum=0

local function calcChecksum(val)
  return milchecksum + val
end

-- Item in the current list, where the page break mark should be inserted
function printPageBreak(item,page)
  local hi = node.new("glue")
  local mynode=mknodes(hi,"//["..page.."]")
  local pp=item.next
  item.next=hi
  node.tail(hi).next=pp
end

local milposition=1
local mcheck=0
function insertPageBreaks(head,group)
  t=getMilestones()
  for item in node.traverse_id(glyph, head) do
    if milposition > #t then break end  
      if item.id == glyph and item.char ~= 45 then
        -- Nodes created with function mknodes() have attribute 224 set to 33, 
        -- we must skip them
        if node.has_attribute(item,224)~=33 then
          mcheck=mcheck+item.char
        end
        if mcheck==t[milposition]["checksum"] then
          texio.write_nl("Checksum equality: "..mcheck.." page: "..t[milposition]["page"])       
          printPageBreak(item,t[milposition]["page"])
          milposition=milposition+1
        end
      end
  end
  texio.write_nl("Current checksum: "..mcheck)
  return head
end

local function processLine(line)
  local lcontents=""
  for item in node.traverse(line.list) do
        if item.id == glyph and item.char ~= 45 then
          lcontents = lcontents .. unicode.utf8.char(item.char) 
          milchecksum= calcChecksum(item.char)
        end
  end
  return lcontents
end

function getPageBreaks(head)
  local lcontents = ""
  for line in node.traverse_id(node.id("hlist"), head) do
    lcontents=processLine(line)
    --texio.write_nl(lcontents)
  end
  addMilestone(pagenumber,milchecksum,lcontents)
  return true
end

local miloutput=""
function addMilestone(page,checksum,text)
  if text ~= "" then 
    miloutput = miloutput .. unicode.utf8.format("milestone{\n  page=\"%s\",\n  checksum=%d,\n  text=\"%s\"\n}\n",page,checksum,text)
    texio.write_nl(unicode.utf8.format("Page:%s checksum:%d text:%s",page,checksum,text))
  end
end

function writeMilestones()
  if(miloutput~="") then
    local f=io.open(milestonefile,"w")
    f:write(miloutput)
    f:close()
  else 
    texio.write("Package milestone: No milestones writen to the output")
  end
end


function import(name)
  local f,e = loadfile(name)
  if not f then error(e, 2) end
  setfenv(f, getfenv(2))
  return f()
end


function getMilestones()
    local t={}
    local i=1
    function milestone(b)
      --local function injectElements(el) t[#t+1][el]=b[el] end
      t[i]={}
      for k,v in pairs(b) do t[i][k] = v end
      i=i+1
    end 
    import(milestonefile)
    return t
end

function printMilestone(head,current,text)
  local message = mknodes("//["..text.."]")
  return node.insert_after(head,current,message)
end

function mknodes(head, text )
  local current_font = font.current()
  local font_parameters = font.getfont(current_font).parameters
  local n,  last
  -- we should insert the paragraph indentation at the beginning
  --[[
  head = node.new("glue")
  head.spec = node.new("glue_spec")
  head.spec.width = 5 * 2^16
  --]]
  last = node.tail(head)
  local count=0
  for s in string.utfvalues( text ) do
    local char = unicode.utf8.char(s)
    if unicode.utf8.match(char,"%s") then
      -- its a space
      n = node.new("glue")
      n.spec = node.new("glue_spec")
      n.spec.width   = font_parameters.space
      n.spec.shrink  = font_parameters.space_shrink
      n.spec.stretch = font_parameters.space_stretch
    else -- a glyph
      count=count+s
      n = node.new("glyph")
      n.font = current_font
      n.subtype = 1
      n.char = s
      n.lang = tex.language
      n.uchyph = 1
      n.left = tex.lefthyphenmin
      n.right = tex.righthyphenmin
    end
    node.set_attribute(n,224,33)
    last.next = n
    last = n
  end

  -- just to create the prev pointers for tex.linebreak
  node.slide(head)
  return head,count
end

local function printNodeList(head)
  for ii in node.traverse(head) do
    if ii.id == glyph then texio.write(string.char(ii.char)) 
    elseif ii.id == node.id("glue") then texio.write(" ")
    end
  end
end

Answer 2

(This is a suggestion more than an answer to your actual question, but given that it is tricky and so far unanswered…)

To my knowledge, there is no option in TeX4ht that would allow you to directly output the page numbers as anchors so, yes, it would probably have to be done in LaTeX and this is, most likely, not trivial.

Let's face it, page numbers simply do not make sense in HTML and ebook formats, which do not have a physical notion of "page". Inserting the said numbers so the reader can use the document as if it were a PDF is indeed good, but semantically awkward. A possible way around that (i.e. if your editor's style or the subject you are writing about allows it) would be to use something other than page numbers – for instance, it is customary in some fields to number paragraphs throughout the document and to refer to them instead of pages. How to do that would be off topic, but it is not very difficult (a counter, a macro and proper referencing commands).