How can I upgrade Forest code for={Lu}{} so that my diagram compiles with current Forest?

I created the following diagram using Forest version 1:

\documentclass[tikz,border=10pt]{standalone}
\usepackage{forest-1}% forest v.1

\begin{document}

\begin{forest}
for tree={
font=\sffamily\scriptsize,
line width=1pt,
draw,
anchor=west,
child anchor=west,
parent anchor=east,
grow'=east,
align=center,
edge path={
\noexpand\path[line width=1pt, ->, \forestoption{edge}]
(!u.parent anchor) -| +(5pt,0) -- +(5pt,0) |- (.child anchor) \forestoption{edge label};
},
for={Lu}{if={n_children==1}{l sep+=10pt}{}},
},
where level={1}{s+=-20pt}{},
[Here\\Village, name=pc
]
[, phantom
[Bananas, name=fb, xshift=-15pt, where n children=0{tier=sk}{}
[Pineapple, name=sf
[Middling\\villages?, name=solar
[Current\\paper, name=cp, edge label={node [font=\sffamily\scriptsize, near end, auto, xshift=-1.5pt] {Imprudent}}
]
]
[Creative\\endearments?, name=apple
[Fantastic\\imaginations, name=su, edge label={node [font=\sffamily\scriptsize, near end, auto, xshift=-1.5pt] {Basic}}
]
]
[Heathen\\hopscotch?, name=pear
[Beneficent\\benevolence, name=cl, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Higher}}, edge={draw=none}
]
]
]
[Zoological, name=gf
[Orange\\kangaroos?, name=star
[Strawberry\\seeker, name=rs, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Lower}}
]
]
[Fragility of\\foundations?, name=hercules
[Memorial\\cooks, name=ws, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Earthling}}
]
]
[Best\\antelopes?, name=astro
[Stellar\\horizons, name=ca, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Martian}}
]
]
]
]
]
]
]
\begin{scope}[line width=1pt, ->]
\draw (solar.east) -- (solar.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (su.west);
\draw (apple.east) -| +(5pt,0) -- +(5pt,0) |- (cp.west);
\draw (pear.east) -- (pear.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cp.west);
\draw (pear.east) -- (pear.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (su.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ws.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ca.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (rs.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ws.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (rs.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ca.west);
\draw (avid.north) -- (avid.north |- fb.south);
\draw (arid.south -| avid.north) -- (avid.north |- fb.north);
\end{scope}
\end{forest}
\end{document}


This code will not compile with v.2 or newer of Forest because

    for={Lu}{if={n_children==1}{l sep+=10pt}{}},


is no longer valid code: for was removed by the upgrade.

The manual tells me that the key has been renamed to for group (p. 85):

These keys have been renamed:

old     new           compat key (all but the last are in -most)
...
for     for group     1.0-for
...


I ended up omitting this diagram from my slides altogether together today because I couldn't figure out how to update this for current Forest from the instructions in the manual.

Initially, I tried,

    for group={Lu}{if={n_children==1}{l sep+=10pt}{}},


However, this did not go well:

! Package pgfkeys Error: I do not know the key '/tikz/{Lu}{if={n_children==1}{l
sep+=10pt}{}}' and I am going to ignore it. Perhaps you misspelled it.

See the pgfkeys package documentation for explanation.


I take it this means that Forest parses for group= but then tries to pass {Lu}{if ...} to TikZ as a single key. I'm not sure why. I'd expect it to look for something to group and so to get some sort of error from Forest, if nothing else.

Clicking on the link for information about for group, I end up on page 30 with information about for <step>. Since this includes examples such as

for nodewalk={uu2}{blue}


and

for n=2{circle,draw}


it is not obvious what I'm meant to do differently with for group.

The group step is documented on page 50-1.

step group=<nodewalk>

Treat <nodewalk> as a single step of the (outer) nodewalk, i.e. the outer every-step keylist is executed only at the end of the embedded nodewalk. The embedded <nodewalk> inherits history from the outer nodewalk. Using this key is equivalent to writing

Nodewalk={every step=independent,history=inherited}<nodewalk>{}, current


→ It is safe to modify the every-step keylist of <nodewalk>.

→ for group is also defined.

Now, I must admit that embedded node walks remain a bit of a mystery to me and my heart tends to sink when I find what I think I need in this part of the manual.

If group is defined in this way, however, I'm not sure how for group can make sense, since for Nodewalk doesn't seem to make sense. That is, it seems that group can only be a step in an inner, rather than an outer, nodewalk.

I therefore try

    for nodewalk={group={Lu}}{if={n_children==1}{l sep+=10pt}{}},


This tries to use group as a step in an inner, embedded nodewalk. This will cause the last step to become the current step in the outer nodewalk, if I've understood correctly, and my conditional should then be applied in, I think, the same way as the original code in version 1.

However, I get another error

! Package forest Error: nodewalk stepped to the invalid node
(forest)                nodewalk stack: ",root',tree,group,last leaf".

See the forest package documentation for explanation.


This is a Forest error, which is better, but I'm not sure I understand it. It looks as if the L is occurring in the outer nodewalk, after group. But Lu is supposed to be the group, so I'm not sure what this means.

However, this might simply be due to another change from version 1 to 2 of Forest: namely, that a non-existent step now causes an error in cases where it did not used to do so.

I try

    for nodewalk={on invalid=fake,group={Lu}}{if={n_children==1}{l sep+=10pt}{}},


However, I can't use on invalid in an outer nodewalk in this way. I don't completely understand why, but I do know it doesn't work.

I therefore try

    for nodewalk={%
Nodewalk={on invalid=fake}{group={Lu}}{if={n_children==1}{l sep+=10pt}{}}
}{},


which gives me

! Package forest Error: nodewalk stepped to the invalid node
(forest)                nodewalk stack: ",root',tree,Nodewalk,group,last leaf".


Thinking that I may need the on invalid inside the group, I try

    for nodewalk={%
Nodewalk={on invalid=fake}{group={on invalid=fake,Lu}}{if={n_children==1}{l sep+=10pt}{}}
}{},


which complains that f is not a valid choice for on invalid again.

Giving up, I change my call line to

\usepackage[compat=1.0-for]{forest}


and reinstate

for={Lu}{if={n_children==1}{l sep+=10pt}{}},


This gives me the complaint about invalidity again:

! Package forest Error: nodewalk stepped to the invalid node
(forest)                nodewalk stack: ",root',tree,last leaf".


So I try

\usepackage[compat=1.0-for,compat=1.0-forstep]{forest}


which gives me an error I have no idea what to do with

! Undefined control sequence.
\forest@forstepwrapper ...dewalk@config@oninvalid
}}\forest@marshal


I give up and cut the diagram out of my presentation for class.

If I use

\usepackage[compat=1.0-all]{forest}


then my code does compile with version 2, but it looks nothing like the result obtained with version 1:

which is hardly what one might hope a compatibility key would do!

On the other hand, using compat=1.0-most still produces the invalidity error.

How should the upgrade instructions be applied in this case and how can that be determined from the list of changes or other revisions in the manual?

Why do the keys for compatibility fail here and why so spectacularly in the case of compat=1.0-all?

First a quick answer, just to make the example compile. Analysis below.

\makeatletter
\def\forest@nodewalk@config@oninvalid{fake}
\makeatother
\usepackage[compat={1.0-for,1.0-forstep}]{forest}


and replace for={Lu}{...} with for={last leaf',u}. The result will not be exactly the same, but close enough.

Quick edit 1, fixing the l distance of the leaves on the right. Replace for={Lu}{if={n_children==1}{l sep+=10pt}{}} by for last leaf={for parent={if={n_children==1}{l sep+=10pt}{}}}. (This still requires compat=1.0-forstep.)

Quick edit 2, fixing the s distance of the root's children. Replace where level={1}{s+=-20pt}{} by s sep+=20pt.

The complete v2 code yielding exactly the same result as v1:

\documentclass[tikz,border=10pt]{standalone}
\makeatletter
\def\forest@nodewalk@config@oninvalid{fake}
\makeatother
\usepackage[compat=1.0-forstep]{forest}

\begin{document}

\begin{forest}
for tree={
font=\sffamily\scriptsize,
line width=1pt,
draw,
anchor=west,
child anchor=west,
parent anchor=east,
grow'=east,
align=center,
edge path={
\noexpand\path[line width=1pt, ->, \forestoption{edge}]
(!u.parent anchor) -| +(5pt,0) -- +(5pt,0) |- (.child anchor) \forestoption{edge label};
},
% for={last leaf',u}{if={n_children==1}{l sep+=10pt}{}},
for last leaf={for parent={if={n_children==1}{l sep+=10pt}{}}},
},
% where level={1}{s+=-20pt}{}
s sep+=20pt,
[Here\\Village, name=pc
]
[, phantom
[Bananas, name=fb, xshift=-15pt, where n children=0{tier=sk}{}
[Pineapple, name=sf
[Middling\\villages?, name=solar
[Current\\paper, name=cp, edge label={node [font=\sffamily\scriptsize, near end, auto, xshift=-1.5pt] {Imprudent}}
]
]
[Creative\\endearments?, name=apple
[Fantastic\\imaginations, name=su, edge label={node [font=\sffamily\scriptsize, near end, auto, xshift=-1.5pt] {Basic}}
]
]
[Heathen\\hopscotch?, name=pear
[Beneficent\\benevolence, name=cl, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Higher}}, edge={draw=none}
]
]
]
[Zoological, name=gf
[Orange\\kangaroos?, name=star
[Strawberry\\seeker, name=rs, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Lower}}
]
]
[Fragility of\\foundations?, name=hercules
[Memorial\\cooks, name=ws, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Earthling}}
]
]
[Best\\antelopes?, name=astro
[Stellar\\horizons, name=ca, edge label={node [font=\sffamily\scriptsize, near end, xshift=-1.5pt, below] {Martian}}
]
]
]
]
]
]
]
\begin{scope}[line width=1pt, ->]
\draw (solar.east) -- (solar.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (su.west);
\draw (apple.east) -| +(5pt,0) -- +(5pt,0) |- (cp.west);
\draw (pear.east) -- (pear.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cp.west);
\draw (pear.east) -- (pear.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (su.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ws.west);
\draw (star.east) -- (star.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ca.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (rs.west);
\draw (astro.east) -- (astro.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ws.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (cl.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (rs.west);
\draw (hercules.east) -- (hercules.east -| apple.east) -| +(5pt,0) -- +(5pt,0) |- (ca.west);
\draw (avid.north) -- (avid.north |- fb.south);
\draw (arid.south -| avid.north) -- (avid.north |- fb.north);
\end{scope}
\end{forest}
\end{document}


Analysis

OP's brave attempts to fight my package were thwarted by no less than four bugs.

• A bug in compat=1.0-forstep. An internal macro \forest@nodewalk@config@oninvalid, which the compatibility mode is supposed to save and restore, was not initialized during package loading. The workaround is to define this macro before loading the package, as shown at the top of this answer.

• Incomplete implementation of compat=1.0-forstep. While this compat key made forest silently ingore a step (above, L) to an invalid node, making a further step (above, u) resulted in error, unlike in v1. The workaround is given in quick edit 1 above.

• A bug in for group. While group within a nodewalk expects an argument, for group does not. This bug also affects to next on tier, previous on tier (which I have discovered actually contain two more bugs ... )

• A bug in compat=1.0-stages (which is included in compat=1.0-all), introduced in v2.0.2: the given options are not processed at all.

OP made a single mistake in her fight: trying to use on invalid incorrectly. This key takes not one but two arguments: the mode and the keys to be executed in this mode. (But the documentation is partially wrong here, as well!)

Final detail: where level={1}{s+=-20pt}{} worked only due to a bug in v1. The proper way to increase the s-distance between siblings is to increase s sep of their parent, as shown in quick edit 2 above.

To make the code work in v2 without compat, replace the original for={Lu}{...} by

if n children={0}{for parent={if={n_children==1}{l sep+=30pt}{}}}{}

• Thanks! So there's no direct translation of the code .... That's helpful to know. How would you do it without the compat key? The problem is that in many cases, I have trees using the new Forest and maybe a tree written using the old one in the same document. Usually, this is no problem: I just update the old trees as I find them and everything is fine. But it becomes problematic if I need a compat key for some trees and not others. – cfr Nov 29 '16 at 16:28
• I guess compat=1.0-all is just broken right now? Please see chat.stackexchange.com/transcript/message/33812983#33812983. – cfr Nov 30 '16 at 1:07
• Write compat=none and \forestcompat{...} in a TeX group to locally enable the compatibility mode. – Sašo Živanović Dec 1 '16 at 23:50
• Luckily, I have a copy of version 1 and only needed 1 tree in this document. Unluckily, I didn't remember this before class! – cfr Dec 1 '16 at 23:54
• In the original, l sep+=10pt actually gets executed several times for each node, as it is embedded under for tree. – Sašo Živanović Dec 2 '16 at 0:31