How to prevent words from running off the edge in ConTeXt?

Question

I have a document containing columns 25 characters wide. I have set the language to US English, and some hyphenation appears, however, I have observed that in many places, scientific terms are not hyphenated. The columns appear like this:

This is a column with     This is another colu-
some  text.  It looks     umn of text.  Notice
fine most of the time,    how some of the text
but sometimes superscientficom the left column
words   do  not  fit.     also  appears  here?

• The document source is generated from scripts, so I cannot readily manually hyphenate all of these words.

• I have tried all of the available settings in tolerance. \setuptolerance[horizontal,stretch] is the only setting which stops these words from running off the page, but it also produces large spaces between all of the other words, which is not ideal.

E.g.:

This is a column with     This is another colu-
some  text.  It looks     umn of text.  Notice
fine most of the time,    how some of the text
but         sometimes     from    the     left
superscientific words     column also  appears
do       not      fit.    here?

How can I prevent these words from running into the next column, without making other serious problems in the document?

Answer 1

You have a couple of options that might improve your situation.

Increase the inter-word space

One is to tell ConTeXt to prefer a larger inter-word space in favour of words sneaking into the adjacent column. But as you said, this alone might result in unacceptable large gaps.

\setupcolumns [tolerance={verytolerant,stretch}]

Assist TeX with the hyphenation

As you said this is no option for you, since the words are automatically generated, but in general this leads to good hyphenation of uncommon words.

\hyphenation{su-per-scien-ti-fic}

Enable font expansion

This alone might not help much for narrow columns, but in combination with the other methods it might improve the overall result, since TeX can slightly change the width of the individual characters, which results in better line breaking.

\definefontfeature [default] [default] [expansion=quality]
\setupalign [hz]

Use wider columns

This is probably the easiest way to improve the situation. However, often the layout is fixed and cannot be influenced.

Use ragged-right alignment

In ragged-aligned text it's much harder to tell if individual words stick into the margin. But the same applies as to using wider columns. One doesn't always have the freedom to change the layout.

As you see, the is no easy way to do it, especially for automatically generated text. Narrow columns are always tricky. When you have a look at magazines with narrow columns, you will often notice extra wide inter-word spacing (or worse: inter character spacing, which is the worst option of all).

Answer 2

In general: Don’t use columns, avoid unreasonably short \hsizees when typesetting paragraphs. Marco already gave you the answers that may get you a typographically good result.

If this still doesn’t satisfy you need and you cannot simply rewrite the text using shorter words, you might as well consider turning to the dark side™: You can resort to letter spacing, just as newspapers and other junk typography does. The following code -- which I adapted from the glorious chickenize package (see its manual for documentation) -- lets you define environments for different values.

Save this snippet as lsp_adj.lua:

thirddata         = thirddata or { }
thirddata.lsp_adj = { }
local lsp_adj     = thirddata.lsp_adj
lsp_adj.callbacks = { }

local nodecopy                     = node.copy
local nodeid                       = node.id
local nodeinsert_before            = node.insert_before
local nodenew                      = node.new
local nodetraverse_id              = node.traverse_id
local nodesinstallattributehandler = nodes.installattributehandler
local nodestasksappendaction       = nodes.tasks.appendaction
local nodestasksdisableaction      = nodes.tasks.disableaction

--- For tests later.
local GLYPH_NODE = nodeid"glyph"
local DISC_NODE  = nodeid"disc"

lsp_adj.new_callback = function (id, stretch)
  --- Prepare a glue; this will be copied all over.
  local letterspace_glue   = nodenew(nodeid"glue")
  local letterspace_spec   = nodenew(nodeid"glue_spec")
  letterspace_spec.width   = tex.sp"0pt"
  letterspace_spec.stretch = tex.sp(stretch)
  letterspace_glue.spec    = letterspace_spec

  --- We’re gonna need a nobreak style penalty too. Else there will be
  --- line breaks inside words.
  local letterspace_pen    = nodenew(nodeid"penalty")
  letterspace_pen.penalty  = 10000

  --- Generate a callback; note the order of arguments.
  local cbk = function (_, _, head)
    for glyph in nodetraverse_id(GLYPH_NODE, head) do
      --- Insert glue between glyphs and ligatures.
      local prev = glyph.prev
      if prev and prev.id == GLYPH_NODE or prev.id == DISC_NODE then
        local g = nodecopy(letterspace_glue)
        nodeinsert_before(head, glyph, g)
        --- Don’t allow breaks here.
        nodeinsert_before(head, g, nodecopy(letterspace_pen))
      end
    end
    return head
  end

  --- Registering a callback requires some verbosity in Context.
  --- But it’s well worth it ...
  local cbk_id = "lsp_" .. id
  lsp_adj.callbacks[id] = nodesinstallattributehandler{
    name        = cbk_id,
    namespace   = thirddata.lsp_adj,
    processor   = cbk,
  }
  nodestasksappendaction(
    "processors",
    -- This one is for users according to node-tsk.lua.
    "before",
    "thirddata.lsp_adj.callbacks." .. id
  )
  nodestasksdisableaction(
    "processors",
    "thirddata.lsp_adj.callbacks." .. id
  )
end

And test it with this code:

\ctxloadluafile{lsp_adj.lua}{}
\dontcomplain

\setuplayout[middle]
\setuplayout[backspace=1cm,topspace=1cm]

\unprotect

%%% Firstly, set some grotesquely unesthetic default.
\newdimen\letter_space_stretch \letter_space_stretch=2pt

\def\defineletterspace{\dodoubleempty\do_define_letter_space}

\def\do_define_letter_space[#1][#2]{%
  \begingroup
  \edef\id{#1}%
  \ifsecondargument
    \ctxlua{
      thirddata.lsp_adj.new_callback(\!!bs\id\!!es, \!!bs#2\!!es)
    }%
    \expandafter\gdef\csname start\id\endcsname{%
      \ctxlua{
        nodes.tasks.enableaction(
          \!!bs processors\!!es,
          \!!bs thirddata.lsp_adj.callbacks.#1\!!es
        )
      }%
    }%
    \expandafter\gdef\csname  stop\id\endcsname{%
      \ctxlua{
        nodes.tasks.disableaction(
          \!!bs processors\!!es,
          \!!bs thirddata.lsp_adj.callbacks.#1\!!es
        ) 
      }%  
    }%  
  \fi 
  \endgroup%
}   

\protect

%%% Now we can define our environments for testing.  Note that this is
%%% no module so you won’t get corresponding setups.
\defineletterspace [smalllsp]  [.5pt]
\defineletterspace [mediumlsp] [1pt]
\defineletterspace [biglsp]    [2pt]
\starttext

%%% For reference: typeset the demo text without letterspacing.
\startcolumns[n=4]
  \input knuth
\stopcolumns

\hairline

\startsmalllsp
  \startcolumns[n=4]
    \input knuth
  \stopcolumns
\stopsmalllsp

\hairline

\startbiglsp
  \startcolumns[n=4]
    \input knuth
  \stopcolumns
\stopbiglsp

\hairline

\startmediumlsp
  \startcolumns[n=4]
    \input knuth
  \stopcolumns
\stopmediumlsp

\stoptext

The result won’t please esthetically but depending on the letterspacing factor it may fulfill your constraints.