Ignore spaces, including ~

Question

The title says it all, how can I ignore spaces, like \ignorespaces does, but including a ~ ?

The reason I ask is we are multiple authors writing a document, and they have different habits with respect to typing French guillemets in their source code (ie. «bla», « bla » and «~bla~»), and I want to unify this by setting a \newunicodechar{«} with the appropriate definition.

For the closing guillemet, \unskip seems to do the trick in all cases.

Answer 1

Here's a LuaLaTeX-based solution. It defines a Lua function that does most of the work, plus a couple of LaTeX utility macros that activate and deactivate the Lua function. By "activate", I mean "assign the Lua function to LuaTeX's process_input_buffer callback", so that it may act as a preprocessor on the input stream before TeX starts its usual processing.

% !TEX TS-program = lualatex
\documentclass{article}
\usepackage[french]{babel} % for "\og" and "\fg" macros
\usepackage[french=guillemets]{csquotes} % for "\enquote" macro

\usepackage{luacode} % for "luacode" environment
%% Lua-side code
\begin{luacode}

function delete_whitespace ( s )
  s = s:gsub ( "«[ ~]*" , "\\og " )
  s = s:gsub ( "[ ~]*»" , "\\fg " )
  -- s = s:gsub ( "[ ~]+([%:%;%?%!])" , "%1" ) -- if needed
  return s
end

\end{luacode}
%% LaTeX-side code
\newcommand\DeletewhitespaceOn{\luadirect{luatexbase.add_to_callback (
    "process_input_buffer", delete_whitespace , "deletewhitespace" )}}
\newcommand\DeletewhitespaceOff{\luadirect{luatexbase.remove_from_callback (
    "process_input_buffer", "deletewhitespace" )}}
\AtBeginDocument{\DeletewhitespaceOn} % enable by default

\begin{document}
\enquote{bla} \og{}bla\fg{}  «bla»  « bla »  «~bla~»  «~ bla ~ » 

\DeletewhitespaceOff
\enquote{bla} \og{}bla\fg{}  «bla»  « bla »  «~bla~»  «~ bla ~ » 
\end{document}

Answer 2

With expl3 it's really easy (although because of the extreme generality of the relevant functions, the performance might not be ideal):

%! TEX program = lualatex
\documentclass{article}
\usepackage{newunicodechar}

\ExplSyntaxOn
\newunicodechar{×}{123\ignorespaces}
\newunicodechar{≡}{123\peek_regex_remove_once:nT{(\cA\~|\cS\ )+}{}}
\ExplSyntaxOff

\begin{document}



× 456

×~456 %unfortunately does not work

% all of the below works:
≡ 456

≡~456

≡~~456

≡~ ~ 456

\end{document}

Just to demonstrate here I use 2 irrelevant Unicode characters.

Performance could be optimized a bit by precompiling the regex:

\regex_new:N \l_ysalmon_regex
\regex_set:Nn \l_ysalmon_regex {(\cA\~|\cS\ )+}
\newunicodechar{≡}{123\peek_regex_remove_once:NT\l_ysalmon_regex{}}

(variable named according to the OP's username. Change if needed)

The peek family of functions does not handle some corner cases correctly, but it's so rare it's virtually impossible to come up in practice.

Answer 3

It's easy to remove all glue, kerns and penalties before a closing guillemet than to remove what's after an opening one.

Anyway, this should be fairly efficient.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{newunicodechar}

\newunicodechar{«}{<<\ignoreallspaces}
\newunicodechar{»}{\removeallspaces~>>}

\ExplSyntaxOn
\NewDocumentCommand{\removeallspaces}{}
 {
  \int_case:nnT { \lastnodetype }
   {
    {11}{\unskip}
    {12}{\unkern}
    {13}{\unpenalty}
   }
   {\removeallspaces}
 }

\NewDocumentCommand{\ignoreallspaces}{}
 {
  \peek_remove_filler:n { \peek_charcode_remove:NT \c_tilde_str { \ignoreallspaces } }
 }

\ExplSyntaxOff

\begin{document}

« ~ a ~~ »

\end{document}

Answer 4

I would like to mention that babel-french v3.5o fixes the issue (for the LuaTeX engine only): coding «bla» or « bla » or «~bla~» produces the same output.

\documentclass{article}
\usepackage{fontspec}
\usepackage[french]{babel}
\frenchsetup{og=«, fg=»}
\begin{document}
«bla»  « bla »  «~bla~» \frquote{bla}
\end{document}

prints