LuaLaTeX: Is shell-escape not required? Does it really make lualatex more vulnerable as an application?

Question

Few weeks ago when I did a fresh install of MacTeX 2020 and tried compiling some documents with command >>lualatex filename.tex I would get this error: attempt to call a nil value (field 'cpath specification'). Upon copy-pasting that error in browser, I got to this post that seemed to suggest that since TeXLive 2020, lualatex needs to be run with --shell-escape. Though unlike the case there, I did not have any dynamic library that I was trying to load (the error occurred with pretty much any file, though all my lualatex files required an external .lua file which also didn't in-turn require any dynamic library). At that time I tried using --shell-escape and it had resolved the problem. Since then I have updated all packages once, and it seems like I no longer need --shell-escape for compiling same documents. Did something change? the release notes don't have anything new, or am I looking at the wrong release notes?

As part 2 of the question: Does --shell-escape make LuaLaTeX users more vulnerable to malicious code? Related to this: Now tex live utility asks a one time question to enhance security by clicking a dialog, what does it actually do?

Answer 1

Recently there was a lualibs bug which caused that error to be printed whenever a module couldn't be loaded using the primary mechanism when shell-escape wasn't enabled, breaking some code relying on alternative loading locations. Yesterday this bug got fixed, which is probably why your code started working without shell-escape.

As long as you know which code is executed, using shell-escape for some purpose is fine. But since shell-escape allows to run arbitrary native binaries, running untrusted code with shell-escape enabled is basically the same as downloading untrusted applications and executing them without any restrictions. In this case, untrusted code includes code in packages, so even if you write your document yourself but use untrusted packages, these cas run arbitrary code. So especially when you don't know why your document should require shell-escape, you should probably keep it disabled.

Answer 2

Most LuaLaTeX documents do not require --shell-escape. I’ve compiled many documents with LuaLaTeX in TeX Live 2020 on Windows and Linux without the need for them. One package that does require shell escapes is minted. So do texlua install scripts.

It’s impossible to say what in the document required a shell escape without a MWE.

Shell escapes are a security risk in general, because they can run arbitrary code. On Linux, I try to mitigate this by installing all TeX files as a system user with access to nothing but the TeX distribution. All scripts therefore run as that user, not as my user account or root.

However, that’s really of little practical value. A script that can modify the TeX installation can modify programs, which I then run with the security credentials to do basically everything but install drivers.