Spurious newline inserted by lstlisting when using the font BQN386 [duplicate]

Question

I have a listing rendering problem with source code from the BQN programming language. I use lualatex (LuaHBTeX, Version 1.17.0 (TeX Live 2023)). I tested various fonts and the result is the same. The fonts come from https://mlochbaum.github.io/BQN/fonts.html and in particular BQN386 is at https://github.com/dzaima/BQN386.

A spurious newline is inserted before 8 by lstlisting and the space before ↕ has disappeared. The same line using the verbatim environment or with verb is ok. Note that I cannot use the command lstinline because of an error. It looks like those special characters are messing with how listings interprets them.

Here is the relevant part of the generated pdf:

I have checked Latex: Listings with monospace fonts, How to enable contextual substitutions in lstlisting? and Use JetBrains Mono ligatures with fontspec and lstlisting environment.

Any idea to render the source code properly ?

MWE:

\documentclass{article}

\usepackage{fontspec}
\setmonofont{BQN386}

\usepackage{listings}
% \lstset{inputencoding=utf8,extendedchars=true}
\lstset{
    basicstyle = \ttfamily,
    % columns = flexible,
    % keepspaces = true,
    % alsoletter={↕},
}

% \lstdefinelanguage{BQN}{ %
%   alsoletter=⌽⊸⌈↕, %
% }
% \lstset{language=BQN}

% \makeatletter
% \renewcommand*\verbatim@nolig@list{}
% \makeatother

\begin{document}

OK: {\tt ⌽⊸⌈ ↕8}. OK: \verb+⌽⊸⌈ ↕8+.
% File ended while scanning use of \lst@temp.
% Or \lstinline!⌽⊸⌈ ↕8!.

\par OK:
\begin{verbatim}
⌽⊸⌈ ↕8
\end{verbatim}

\par Not OK:
\begin{lstlisting}
⌽⊸⌈ ↕8
\end{lstlisting}

\end{document}

Answer 1

You need to setup the characters explicitly for listings. For APL you can use https://tex.stackexchange.com/a/25796/. The character ⊸ (0x22b8) is missing there, I have added that one below. Now also \lstinline works properly.

\documentclass{article}

\usepackage{fontspec}
\setmonofont{BQN386}

\usepackage{listings}

\lstset{
    basicstyle = \ttfamily,
    % columns = flexible,
    % keepspaces = true,
    % alsoletter={↕},
}

% set lstlisting to accept UTF8 APL text
% from: https://tex.stackexchange.com/a/25796/
\makeatletter
\lst@InputCatcodes
\def\lst@DefEC{%
 \lst@CCECUse \lst@ProcessLetter
  ^^80^^81^^82^^83^^84^^85^^86^^87^^88^^89^^8a^^8b^^8c^^8d^^8e^^8f%
  ^^90^^91^^92^^93^^94^^95^^96^^97^^98^^99^^9a^^9b^^9c^^9d^^9e^^9f%
  ^^a0^^a1^^a2^^a3^^a4^^a5^^a6^^a7^^a8^^a9^^aa^^ab^^ac^^ad^^ae^^af%
  ^^b0^^b1^^b2^^b3^^b4^^b5^^b6^^b7^^b8^^b9^^ba^^bb^^bc^^bd^^be^^bf%
  ^^c0^^c1^^c2^^c3^^c4^^c5^^c6^^c7^^c8^^c9^^ca^^cb^^cc^^cd^^ce^^cf%
  ^^d0^^d1^^d2^^d3^^d4^^d5^^d6^^d7^^d8^^d9^^da^^db^^dc^^dd^^de^^df%
  ^^e0^^e1^^e2^^e3^^e4^^e5^^e6^^e7^^e8^^e9^^ea^^eb^^ec^^ed^^ee^^ef%
  ^^f0^^f1^^f2^^f3^^f4^^f5^^f6^^f7^^f8^^f9^^fa^^fb^^fc^^fd^^fe^^ff%
  ^^^^20ac^^^^0153^^^^0152%
  ^^^^20a7^^^^2190^^^^2191^^^^2192^^^^2193^^^^2206^^^^2207^^^^220a%
  ^^^^2218^^^^2228^^^^2229^^^^222a^^^^2235^^^^223c^^^^2260^^^^2261%
  ^^^^2262^^^^2264^^^^2265^^^^2282^^^^2283^^^^2296^^^^22a2^^^^22a3%
  ^^^^22a4^^^^22a5^^^^22b8^^^^22c4^^^^2308^^^^230a^^^^2336^^^^2337^^^^2339%
  ^^^^233b^^^^233d^^^^233f^^^^2340^^^^2342^^^^2347^^^^2348^^^^2349%
  ^^^^234b^^^^234e^^^^2350^^^^2352^^^^2355^^^^2357^^^^2359^^^^235d%
  ^^^^235e^^^^235f^^^^2361^^^^2362^^^^2363^^^^2364^^^^2365^^^^2368%
  ^^^^236a^^^^236b^^^^236c^^^^2371^^^^2372^^^^2373^^^^2374^^^^2375%
  ^^^^2377^^^^2378^^^^237a^^^^2395^^^^25af^^^^25ca^^^^25cb%
  ^^00}
\lst@RestoreCatcodes
\makeatother


% \lstdefinelanguage{BQN}{ %
%   alsoletter=⌽⊸⌈↕, %
% }
% \lstset{language=BQN}

% \makeatletter
% \renewcommand*\verbatim@nolig@list{}
% \makeatother

\begin{document}

OK: {\tt ⌽⊸⌈ ↕8}. OK: \verb+⌽⊸⌈ ↕8+.

Or \lstinline!⌽⊸⌈ ↕8!.

\par OK:
\begin{verbatim}
⌽⊸⌈ ↕8
\end{verbatim}

\par Not OK:
\begin{lstlisting}
⌽⊸⌈ ↕8
\end{lstlisting}

\end{document}