6

I am trying to access SQLite databases from LuaLaTeX but am not able to load an installed lsqlite3 package. Here's what I tried so far:

  1. Took a brandnew Xubuntu 14.04 installation
  2. Installed a complete TeX Live 2014
  3. Installed Luarocks (which installed Lua5.1 on the way)
  4. Used Luarocks to install lsqlite3

    uwe@luabuntu:/media/uwe$ luarocks list

    Installed rocks:

    lsqlite3 0.9.1-2 (installed) - /usr/local/lib/luarocks/rocks

  5. Appended the package.path in the following TeX file:

    \documentclass{article}    
    \usepackage{luacode}
    
    \begin{document}
    
    \begin{luacode*}
      package.path="/usr/local/lib/luarocks/rocks;/usr/local/lib/luarocks/rocks/lsqlite3; 
      /usr/local/lib/lua/5.1/lsqlite3.so" .. package.path
    
      require("lsqlite3")
    \end{luacode*}
    
    Hello Lua!
    
    \end{document}
    

The error I receive is the following:

! LuaTeX error [\directlua]:3: module 'lsqlite3' not found:
  no field package.preload['lsqlite3']
  [luatexbase.loader] Search failed
  [kpse lua searcher] file not found: 'lsqlite3'
  [kpse C searcher] file not found: 'lsqlite3'
  [oberdiek.luatex.kpse_module_loader]-eroux Search failed
stack traceback:
  [C]: in function 'require'
  [\directlua]:3: in main chunk.
\luacode@dbg@exec ...code@maybe@printdbg {#1} #1 }

l.9 \end{luacode*}

What must be set to have LuaLaTeX find the lsqlite3 package?

EDIT

I have tried to adjust everything following the comments below:

  1. Adjusted /usr/local/texlive/2014/texmf.cnf
  2. Put everything into the example
  3. Still get the error about the missing lsqlite3 module with the following example

    \documentclass{article}
    \directlua{
    require "lualoader"
    }
    
    \usepackage{fontspec}
    \usepackage[english]{babel}
    \begin{document}
    
    \directlua{package.path='/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/home/uwe/.luarocks/share/lua/5.1/?.lua;/home/uwe/.luarocks/share/lua/5.1/?/init.lua;/usr/share/lua/5.1//?.lua;/usr/share/lua/5.1//?/init.lua;./?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/lib/lua/5.1/?.lua;/usr/local/lib/lua/5.1/?/init.lua;/usr/share/lua/5.1/?.lua;/usr/share/lua/5.1/?/init.lua'
    package.cpath='/usr/local/lib/lua/5.1/?.so;/home/uwe/.luarocks/lib/lua/5.1/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/lib/x86_64-linux-gnu/lua/5.1/?.so;/usr/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so'
    require "lualoader"
    }
    
    \directlua{%
    local sqlite3 = require("lsqlite3")
    local db = sqlite3.open_memory()
    
    db:exec[[
      CREATE TABLE test (id INTEGER PRIMARY KEY, content);
    
      INSERT INTO test VALUES (NULL, 'Hello World');
      INSERT INTO test VALUES (NULL, 'Hello Lua');
      INSERT INTO test VALUES (NULL, 'Hello Sqlite3')
    ]]
    
    for row in db:nrows("SELECT * FROM test") do
      tex.print(row.id .. " : ".. row.content )
    end
    
    }
    \end{document}
    
  • 1
    lsqlite3 is a binary module, at least in part. You probably have to set package.cpath as well. – Philipp Gesang Dec 21 '14 at 14:35
  • 1
    I have the strong feeling that package.path is ignored in kpathsea systems. - see tex.stackexchange.com/a/31904/243 – topskip Dec 21 '14 at 17:11
  • 1
    See also tex.stackexchange.com/q/48193/243 – topskip Dec 21 '14 at 17:13
  • 1
    @UweZiegenhagen you call lualoader twice in your edit, paths must be set before it is required – michal.h21 Dec 25 '14 at 18:44
  • 1
    The interpreted parts can be configured by means of the environment variable LUAINPUTS (example: export LUAINPUTS="~/code/tex/pgfplots/tex//:~/code/tex/pgf/generic//:"). There is also an environment variable CLUAINPUTS ... perhaps you need to adopt it? I know that DLL search paths tend to need native load procedures, in other words: environment variables (don't know how that works in LUA, though) – Christian Feuersänger Dec 25 '14 at 19:55
11
+100

It is possible to modify package loading mechanism in Lua, which is the case of LuaTeX, where kpse library is used to load modules instead of default mechanism with package.path and package.cpath. So it doesn't have any effect to set these variables. But nothing prevents us from using both methods, using simple Lua module, lualoader.lua (save this file in directory with your document):

-- this is copied from luatexbase.loader
local make_loader = function(path, pos, loadfunc)
  local default_loader = package.searchers[pos]
  local loader = function(name)
    local file, _ = package.searchpath(name,path)
    if not file then
      local msg = "\n\t[lualoader] Search failed"
      local ret = default_loader(name)
      if type(ret) == "string" then
        return msg ..ret
      elseif type(ret) == "nil" then
        return msg
      else
        return ret
      end
    end
    local loader,err = loadfunc(file, name)
    if not loader then
      return "\n\t[lualoader] Loading error:\n\t"..err
    end
    return loader
  end
  package.searchers[pos] = loader
end

local binary_loader = function(file, name)
  local symbol = name:gsub("%.","_")
  return package.loadlib(file, "luaopen_"..symbol)
end

make_loader(package.path,2,loadfile)
make_loader(package.cpath,3, binary_loader)

we use function make_loader to insert function searching package.path or package.cpath at specified place in package.searchers table. place 2 is for lua files, place 3 for binary modules. we also need to use different functions for actual module loading, it is loadfile for Lua files and little bit more complicated function for binary modules, using package.loadlib.

Now we can try some example from lsqlite3 distribution adapted for LuaTeX:

\documentclass{article}
\directlua{
require "lualoader"
}

\usepackage{fontspec}
\usepackage[english]{babel}
\begin{document}
\directlua{%
local sqlite3 = require("lsqlite3")

local db = sqlite3.open_memory()

db:exec[[
  CREATE TABLE test (id INTEGER PRIMARY KEY, content);

  INSERT INTO test VALUES (NULL, 'Hello World');
  INSERT INTO test VALUES (NULL, 'Hello Lua');
  INSERT INTO test VALUES (NULL, 'Hello Sqlite3')
]]

for row in db:nrows("SELECT * FROM test") do
  tex.print(row.id .. " : ".. row.content )
end

}
\end{document}

and the result:

enter image description here

Edit:

As Uwe pointed out, luarocks installed by Linux package managers may not work correctly, I experienced this on Fedora, so I installed luarocks from sources then. Another solution may be to set package.path and package.cpath to directories listed by command

luarocks path

it is on my system:

$ luarocks path
export LUA_PATH='/home/michal/.luarocks/share/lua/5.2/?.lua;/home/michal/.luarocks/share/lua/5.2/?/init.lua;/usr/local/share/lua/5.2/?.lua;/usr/local/share/lua/5.2/?/init.lua;/usr/share/lua/5.2/?.lua;/usr/share/lua/5.2/?/init.lua;/usr/lib/lua/5.2/?.lua;/usr/lib/lua/5.2/?/init.lua;./?.lua'
export LUA_CPATH='/home/michal/.luarocks/lib/lua/5.2/?.so;/usr/local/lib/lua/5.2/?.so;/usr/lib/lua/5.2/?.so;/usr/lib/lua/5.2/loadall.so;./?.so'

and modified preamble:

\directlua{    
package.cpath = '/home/michal/.luarocks/lib/lua/5.2/?.so;/usr/local/lib/lua/5.2/?.so;/usr/lib/lua/5.2/?.so;/usr/lib/lua/5.2/loadall.so;./?.so'
package.path = '/home/michal/.luarocks/share/lua/5.2/?.lua;/home/michal/.luarocks/share/lua/5.2/?/init.lua;/usr/local/share/lua/5.2/?.lua;/usr/local/share/lua/5.2/?/init.lua;/usr/share/lua/5.2/?.lua;/usr/share/lua/5.2/?/init.lua;/usr/lib/lua/5.2/?.lua;/usr/lib/lua/5.2/?/init.lua;./?.lua'
require "lualoader"    
  • If I simply run the TeX code I get: "! LuaTeX error [\directlua]:1: module 'lualoader' not found: no field package.preload['lualoader'] [kpse lua searcher] file not found: 'lualoader' [kpse C searcher] file not found: 'lualoader' If I save the lua-loader code in the same folder as the TeX code I get an error as well: LuaTeX error [\directlua]:1: module 'luarocks.loader' not found: no field package.preload['luarocks.loader'] [kpse lua searcher] file not found: 'luarocks.loader' [kpse C searcher] file not found: 'luarocks.loader' – Uwe Ziegenhagen Dec 23 '14 at 16:47
  • @UweZiegenhagen luarocks.loader should be available if you have correctly installed luarocks. in this particular case it is not needed, because lsqlite3 is installed to directory with other Lua libraries, but it is needed for Lua modules installed through luarocks – michal.h21 Dec 23 '14 at 18:06
  • I installed luarocks via the Ubuntu package system. If I remove the line for the luarocks.loader I get the the following error again: ! LuaTeX error [\directlua]:1: module 'lsqlite3' not found: no field package.preload['lsqlite3'] – Uwe Ziegenhagen Dec 23 '14 at 18:35
  • @UweZiegenhagen ah, I installed luarocks from the sources, I had problems when it was installed by Fedora. You may need to set package.path and package.cpath in this case – michal.h21 Dec 23 '14 at 18:50
  • @UweZiegenhagen you're welcome :) – michal.h21 Dec 25 '14 at 20:05
5

As of May 2017, there is a package on CTAN called luapackageloader, that allows you to modify the default package searching behavior in LuaTeX.

Once you load luapackageloader, doing a require will first try the default kpse based searcher. If the module isn’t found, it will also try to load the package from Lua’s package.path and package.cpath values.

Usage example

\documentclass{article}    
\usepackage{luacode}
\usepackage{luapackageloader}

\begin{document}

\begin{luacode*}
  package.path="/usr/local/lib/luarocks/rocks;/usr/local/lib/luarocks/rocks/lsqlite3; 
  /usr/local/lib/lua/5.1/lsqlite3.so" .. package.path

  require("lsqlite3")
\end{luacode*}

Hello Lua!

\end{document}

Additional note: you can do eval $(luarocks path) on your shell to set the package path using environment variables which the Lua interpreter will pick up automatically. That way, you do not need to fiddle with the package.path yourself.

  • This is a valid answer, but could you please adapt the code to address the problem posed in the question? – Henri Menke May 29 '17 at 3:59
1

Based on comments on this question, I have the following line in /usr/local/texlive/2014/texmf.cnf:

CLUAINPUTS  = .;$SELFAUTOLOC/lib/{$progname,$engine,}/lua//;/usr/lib/{$progname,$engine,}/lua//

But my luarocks list gives:

Installed rocks:

lsqlite3
   0.9.1-2 (installed) - /usr/lib/luarocks/rocks-5.2

so you may need to adjust the second path according to your installation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.