LuaTeX/PdfTeX: Incorrect transparency for fonts with overlapping glyphs -- achieve effect similar to Adobe InDesign/Tikz in text mode

Question

Update-1: This issue affects all fonts that assume glyph overlap. The 3 broad categories affected are: 1) All fonts for non-Latin scripts that inherently require glyph overlap, 2) Handwriting (cursive) fonts for Latin script, 3) Decoration fonts meant to build complex decorations (like decorative borders/frames) by repeating patterns of glyphs that partially overlap.

Update-2: This question has not been answered yet. After David Purton posted this ‘related answer’ below, the correct understanding is that tikz "transparency group" does apply grouped transparency to node text too. The mistake in my tikz code was that I invoked transparency grouping, but specified opacity at a different scope level. I have updated my original code in question, and tikz output looks good. This means the code needed for transparency grouping in text mode (outside tikz) can be either adapted from tikz codebase, or written by someone who understands PDF XObjects. It is fine even if transparency is applied only to latex's unbreakable boxes like package textpos‘s environment texblock or minipage, or plain tex's boxes (vbox and hbox); that itself will be a big leap. Someone who understands macro \pdfxform from section 8.8 of pdftex user manual might be able to help.

In some scripts it is almost impossible to have non-overlapping glyph components. Thats either because some complex glyphs are themselves composed of multiple -- potentially overlapping -- component glyphs, or by design adjacent glyphs need to overlap to form a word. Both are true for Devanagari, with the latter due to the fact that "overline" of glyphs in a word has to overlap to make it a word (its a script feature in addition to inter-word spacing). These overlaps are (rightly) not visible when text is set in a solid color, and are visible when it is set in a transparent one.

Setting fontspec Opacity for such scripts produces ugly result (making it unusable), see for instance the following image that is Hindi word for male parent (पिताजी) set with Opacity. As you can see, Opacity setting results in darker spots at overlapping areas. This is because text is not grouped before applying transparency (Opacity setting applies transparency at individual glyph level). Tikz doesn't have this problem if one uses its implicit or explicit transparency grouping (tikz output not shown here, it can be seen by compiling code in this question). Below fontspec Opacity output is the output from Adobe InDesign 2020 (I luckily still had access to trial version of InDesign) which looks great (no dark spots on overlap, and result looks uniform). I tested the output of text transparency grouping from both Tikz and Adobe InDesign, text is not converted to path upon grouping (text is copyable/searchable, which is great). From David Purton's analysis Tikz and Adobe InDesign produce similar PDF code.

LuaLaTeX with fontspec and Opacity<1 output:

Adobe InDesign 2020 output: An ideal solution will perform grouped transparency in text mode (its ok if it just works with HarfBuzz), like the way "transparency groups" work for tikz text and graphics. The text should still act like text in pdf, it should be searchable & copyable, and should not add any restrictions in text mode (such transparent text can span paragraphs, etc). If it makes it easier to implement, a solution could restrict itself to applying grouped transparency to package textpos's environment textblock or minipage such that all objects within that environment will be part of transparency group (potentially with a parameter for blend mode, and potentially groups can be nested just like tikz transparency groups, and color commands \color/\textcolor [though nesting would be an added bonus, not the most important]). As a side note, am not looking for a tikz solution (that acts similar to textpos or minipage with grouped transparency), though you are welcome to post a solution that does that for the broader community here (with first line explicitly stating in bold that "This is not a solution to this question, and is being posted here only to showcase what can be achieved with tikz.").[1]

[1] It might sound too strict, but IMO an incorrect answer that seems to partially solve the question might not invite further inquiry/drive towards a correct solution.

As a reference, the screenshot below from Section 17.3.1 of asppdf manual could lead us to a solution. It looks like the solution for the problem in this question would need to implement a transparency grouping like in the bottom-left figure in screenshot. The paragraphs above the four transparency grouping images explain things pretty well. The current transparency implementation functions like top-right image (transparency with no grouping, causing glyphs to composite with each other).

Here's the test code to regenerate the problem of dark areas at overlap. You might have to zoom-in to see the problem on screen, but it will be glaringly visible & distracting in a printed document. Lastly, this is not a font issue, and can be recreated with any existing Devanagari font.

% >> lualatex opacityoverlap.tex
\documentclass{article}
\usepackage{fontspec}
\usepackage{tikz}

\newfontfamily{\devanagarifamtext}{Noto Sans Devanagari}[Script=Devanagari, Scale=1, Renderer=HarfBuzz, Color=FF0000, Opacity=0.25]

\newfontfamily{\devanagarifamtikz}{Noto Sans Devanagari}[Script=Devanagari, Scale=1, Renderer=HarfBuzz, Color=FF0000]

\begin{document}

Text mode: {\devanagarifamtext एक गांव में पिताजी}

Tikz mode: \tikz[baseline,blend group=hue,opacity=0.25]{\node[inner sep=0pt,minimum width=0pt,outer sep=0pt,anchor=base] () {\devanagarifamtikz एक गांव में पिताजी};}

\end{document}

Answer 1

This is not a solution to this question, and is being posted here only to showcase what can be achieved with tikz/pgf (as instructed in question).

Update: PGF only answer

Here's an optimised solution using only pgf. It has less overhead than using the tikz interface. Having looked at this for too long, I think this is the best solution by a long way. It's much simpler to implement, works with other packages, and has a minimal speed hit.

Blend modes seem unreliable across viewers unless you're careful how you lay out your text in the pgfpicture, so I don't think there's anything to be gained by using them.

%! TeX Program = lualatex

\documentclass{article}

\usepackage{xparse}
\usepackage{pgf}

\ExplSyntaxOn

\NewDocumentEnvironment { transparencygroup }
  { m +b }
  {
    \par
    \dim_set_eq:NN \l_tmpa_dim \prevdepth
    \noindent
    \pgfrememberpicturepositiononpagefalse
    \begin { pgfpicture }
      \pgfsetfillopacity { #1 }
      \begin { pgftransparencygroup } [ isolated=false ]
        \pgfpathmoveto { \pgfpointorigin }
        \pgftext [ base ]
          {
            \vbox:n
              {
                \dim_set_eq:NN \prevdepth \l_tmpa_dim
                #2
              }
          }
      \end { pgftransparencygroup }
    \end { pgfpicture }
  }
  { }

\NewDocumentCommand \texttransparencygroup { m m }
  {
    \mode_leave_vertical:
    \hbox:n
      {
        \pgfrememberpicturepositiononpagefalse
        \begin { pgfpicture }
          \pgfsetfillopacity { #1 }
          \begin { pgftransparencygroup } [ isolated=true ]
            \pgfpathmoveto { \pgfpointorigin }
            \pgftext [ base ] { #2 }
          \end { pgftransparencygroup }
        \end { pgfpicture }
      }
  }

\ExplSyntaxOff

\usepackage{fontspec}

\newfontfamily{\devanagari}{Noto Sans Devanagari}[Script=Devanagari, Renderer=HarfBuzz]

\begin{document}

\Huge
English {\devanagari \texttransparencygroup{0.4}{एक गांव में पिताजी}} English

\devanagari\color{red}
\begin{transparencygroup}{0.5}
  एक गांव में पिताजी
\end{transparencygroup}

\vskip-8mm
\color{green}
\begin{transparencygroup}{0.25}
  एक गांव में पिताजी
\end{transparencygroup}

\end{document}

---

TIKZ answer

Your PDF from InDesign is essentially using the same method as tikz. It creates a Form XObject with a transparency group and puts the text in it. Then it inserts this Form XObject into the Page stream. Even if you don't use tikz you'll still have to box your content up into an XObject. The transparency group isn't just a graphics flag you can turn on and off mid-stream.

The trick with tikz seems to be to put just the text in the transparency group:

    \documentclass{article}
    \usepackage{l3pdf}
    \ExplSyntaxOn
    \pdf_uncompress:
    \ExplSyntaxOff
    \usepackage{fontspec}
    \usepackage{tikz}
    \newfontfamily{\devanagari}{Noto Sans Devanagari}[Script=Devanagari, Scale=1, Renderer=HarfBuzz]
    \begin{document}
    \begin{tikzpicture}[opacity=0.5]
      \fill[cyan] (0,0) circle [radius=10pt];
      \begin{scope}[transparency group]
        \node[text=red, font=\devanagari] {एक गांव में पिताजी};
      \end{scope}
    \end{tikzpicture}
    \end{document}

Here's a fuller example, showing how to put a couple of paragraphs of different colours in a transparency group. It also allows you to specify the blend mode. Any background graphics must be in the blend group though. This is not a limitation if you just want transparency.

\documentclass{article}
\usepackage{l3pdf}
\ExplSyntaxOn
\pdf_uncompress:
\ExplSyntaxOff
\usepackage{fontspec}
\usepackage{tikz}
\usepackage{lipsum}
\newfontfamily{\devanagari}{Noto Sans Devanagari}[Script=Devanagari, Scale=1, Renderer=HarfBuzz]
\newcommand*{\tgopacity}{0.5}
\newcommand*{\tgblendmode}{normal}
\newcommand*{\tggraphics}{}
% \begin{transparentgroup}{opacity}{blend mode}{graphics within blend group}
\newenvironment{transparentgroup}[3]{%
  \renewcommand*{\tgopacity}{#1}%
  \renewcommand*{\tgblendmode}{#2}%
  \renewcommand{\tggraphics}{#3}%
  \setbox0=\vbox\bgroup
}{%
  \egroup
  \noindent\begin{tikzpicture}[
      inner sep=0pt, outer sep=0pt, blend group=\tgblendmode
    ]
    \tggraphics
    \pgfresetboundingbox
    \begin{scope}[transparency group, opacity=\tgopacity]
      \node [anchor=north west] {\box0};
    \end{scope}
  \end{tikzpicture}}
\begin{document}
\noindent\tikz[remember picture, overlay] \fill[green, opacity=0.5]
  ([yshift=7cm]current page.center) circle [radius=20pt];%
\begin{transparentgroup}{0.5}{lighten}{
    \fill[green, opacity=0.5] (10mm,-5mm) circle [radius=20pt];
  }
  \textcolor{red}{\devanagari एक गांव में पिताजी}

  \lipsum[1]
\end{transparentgroup}

\textcolor{red}{\devanagari एक गांव में पिताजी}

\lipsum[1]

\end{document}

Answer 2

You can do what you want, but it's quite a pain. The example below has a simple interface to set material in a \vbox and allows you to set the opacity and blend mode.

The usage is:

\begin{transparencygroup}[
    opacity = < Value between 0 and 1 >,
    blend mode = < One of the standard PDF blend modes >
  ]
\end{transparencygroup}

Because of the poor interface for page resources, there can be problems with other packages that add things to the page resources.

This sample code makes no attempt to work with other packages, so it may not work properly if you have other packages loaded that mess with /ExtGState (e.g., tikz, colorspace, transparent, fontspec's Opacity feature, etc.). tikz provides a hook that could be used if you need to use tikz.

%! TeX Program = lualatex

\documentclass{article}
\pagestyle{empty}

% grouped transparency implementation

% Limitations:
%
% 1. Because of the poor interface for page resources, there can be problems
%    with other packages that add things to the /ExtGState dictionary (e.g.,
%    tikz, colorspace, transparent, etc.).
%
% 2. You can't use the Opacity feature of fontspec as this puts the
%    transparency code inside the transparency group, and you'll still get the
%    overlapping glyphs.
%
% 3. Only LuaLaTeX is supported.

\usepackage{l3pdf}
\usepackage{xparse}
\usepackage{everyshi}

\ExplSyntaxOn

\pdf_uncompress:


% l3pdf extensions (lualatex only)

\cs_generate_variant:Nn \pdf_object_new:nn { xn }
\cs_generate_variant:Nn \pdf_object_write:nn { xx }
\cs_generate_variant:Nn \pdf_object_ref:n { e }

\cs_new_protected:Nn \__reportaman_pdf_xform_now:Nnn
  {
    \tex_immediate:D \tex_pdfxform:D
      attr { #2 }
      resources { #3 }
      #1
  }
\cs_generate_variant:Nn \__reportaman_pdf_xform_now:Nnn { Nxx }

\cs_new_protected:Nx \__reportaman_pdf_xform_last:
  {
    \exp_not:N \int_value:w
    \exp_not:N \tex_pdflastxform:D
    \c_space_tl 0 ~ R
  }

\cs_new_protected:Nn \__reportaman_pdf_refxform_last:
  {
    \tex_pdfrefxform:D \tex_pdflastxform:D
  }

\cs_new_protected:Nn \__reportaman_pdf_pageresources_gput_right:nn
  {
    \tex_global:D
    \tex_pdfvariable:D pageresources
    \exp_after:wN
      {
        \tex_the:D \tex_pdfvariable:D pageresources / #1 ~ #2
      }
  }
\cs_generate_variant:Nn \__reportaman_pdf_pageresources_gput_right:nn { nx }

\cs_new_protected:Nn \__reportaman_pdf_literal_direct:n
  {
    \tex_pdfextension:D literal direct
      {
        #1
      }
  }
\cs_generate_variant:Nn \__reportaman_pdf_literal_direct:n { x }

\cs_new_protected:Nn \__reportaman_pdf_save_gs:
  {
    \__reportaman_pdf_literal_direct:n { q }
  }

\cs_new_protected:Nn \__reportaman_pdf_restore_gs:
  {
    \__reportaman_pdf_literal_direct:n { Q }
  }


% grouped transparency back end

\clist_new:N \g__reportaman_ca_clist
\clist_new:N \g__reportaman_bm_clist
\int_new:N \g__reportaman_extgstate_int

\cs_new_protected:Nn \__reportaman_set_pdf_page_resources:
  {
    \clist_remove_duplicates:N \g__reportaman_ca_clist
    \clist_remove_duplicates:N \g__reportaman_bm_clist
    \tl_clear:N \l_tmpa_tl
    \clist_map_inline:Nn \g__reportaman_ca_clist
      {
        \tl_put_right:Nn \l_tmpa_tl
          {
            /reportaman_CA_##1 ~ << ~ /CA ~ ##1 ~ >> ~
            /reportaman_ca_##1 ~ << ~ /ca ~ ##1 ~ >> ~
          }
      }
    \clist_map_inline:Nn \g__reportaman_bm_clist
      {
        \tl_put_right:Nn \l_tmpa_tl
          {
            /reportaman_bm_##1 ~ << ~ /BM ~ [ ~ /##1 ~ ] ~ >> ~
          }
      }
    \clist_gclear:N \g__reportaman_ca_clist
    \clist_gclear:N \g__reportaman_bm_clist
    \exp_args:Nx \pdf_object_if_exist:nT
      {
        reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int
      }
      {
        \pdf_object_write:xx
          { reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int }
          { \l_tmpa_tl }
        \__reportaman_pdf_pageresources_gput_right:nx
          { ExtGState }
          { 
            \pdf_object_ref:e
              {
                reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int
              }
          }
        \int_gincr:N \g__reportaman_extgstate_int
      }
  }

\EveryShipout { \__reportaman_set_pdf_page_resources: }

\keys_define:nn { reportaman }
{
  opacity      .tl_set:N           = \l__reportaman_opacity_tl,
  opacity      .value_required:n   = true,
  blend ~ mode .choice:,
  blend ~ mode .choices:nn         =
    { Normal, Multiply, Screen, Overlay, Darken, Lighten, ColorDodge,
      ColorBurn, HardLight, SoftLight, Difference, Exclusion, Hue, Saturation,
      Color, Luminosity }
    { \tl_set_eq:NN \l__reportaman_blend_mode_tl \l_keys_choice_tl },
  blend ~ mode .value_required:n   = true,
}


% grouped transparency front end
% \begin{transparencygroup}[
%   opacity = < Value between 0 and 1 >,
%   blend mode = < One of the standard PDF blend modes >
% ]
% \end{transparencygroup}
\NewDocumentEnvironment { transparentgroup }
  { o +b }
  {
    \keys_set:nn { reportaman }
      {
        opacity = { 1 },
        blend ~ mode = { Normal }
      }
    \IfValueT { #1 }
      {
        \keys_set:nn { reportaman } { #1 }
      }
    \clist_gput_right:Nx \g__reportaman_ca_clist { \l__reportaman_opacity_tl }
    \clist_gput_right:Nx \g__reportaman_bm_clist { \l__reportaman_blend_mode_tl }
    \vbox_set:Nn \l_tmpa_box
      {
        \__reportaman_pdf_literal_direct:x
          {
            /reportaman_bm_\l__reportaman_blend_mode_tl \c_space_tl gs
          }
        #2
      }
    \exp_args:Nx \pdf_object_if_exist:nF
      {
        reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int
      }
      {
        \pdf_object_new:xn
          { reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int }
          { dict }
      }
    \__reportaman_pdf_xform_now:Nxx
      \l_tmpa_box
      { /Group ~ << ~ /S ~ /Transparency ~ /I ~ true ~ /K ~ false ~ >> }
      { /ExtGState ~
        \pdf_object_ref:e
        {
          reportaman_extgstate_ \int_use:N \g__reportaman_extgstate_int
        }
      }
    \__reportaman_pdf_save_gs:
    \__reportaman_pdf_literal_direct:x
      {
        /reportaman_CA_\l__reportaman_opacity_tl \c_space_tl gs \iow_newline:
        /reportaman_ca_\l__reportaman_opacity_tl \c_space_tl gs
      }
    \__reportaman_pdf_refxform_last:
    \__reportaman_pdf_restore_gs:
  }
  {
  }

\ExplSyntaxOff


% Usage example

\usepackage{xcolor}
\usepackage{fontspec}
\newfontfamily{\devanagari}{Noto Sans Devanagari}[Script=Devanagari, Scale=1, Renderer=HarfBuzz]

\begin{document}
\begin{transparentgroup}[opacity=0.25, blend mode=Hue]
  \devanagari\Huge
  \color{red}एक गांव में पिताजी

  \vspace{-7mm}\quad
  \color{green}एक गांव में पिताजी
\end{transparentgroup}

\begin{transparentgroup}[opacity=0.5]
  \devanagari\Huge
  \color{red}एक गांव में पिताजी

  \vspace{-7mm}\quad
  \color{blue}एक गांव में पिताजी
\end{transparentgroup}

\begin{transparentgroup}
  \devanagari\Huge
  \color{red}एक गांव में पिताजी

  \vspace{-7mm}\quad
  \color{yellow}एक गांव में पिताजी
\end{transparentgroup}
\end{document}