11

I have a Tex document that uses the listings package to print a Python source code. I also want to separately print certain Python function signatures in my document. I do not want to use the line number functionality from the listings package since the Python code might be edited in the future.

Is there any way to automatically sync certain parts of the Python code with the Tex document (without using line numbers)?

Thank you very much!

  • So you would like to use something like this pseudo code: \lstinputlisting[language=Python,include=myfunction]{filename.py} Where include=... is some function that finds the line numbers for def myfunction(...): and the last line in the function and then expands to firstline=#,lastline=#? Interesting question! – Grimler Jul 18 '16 at 14:19
  • Yes, that is exactly what I'm looking for. – Mandar T Jul 19 '16 at 9:40
5
+50

The main problem is that you need to analyze the Python source code, which TeX can't (easily) do. Luckily Python is capable of analyzing itself, using abstract syntax trees (see also some more thorough documentation). The idea is that we have the Python ast analyze the code and write what it finds to a TeX file.

#!/usr/local/bin/python3

import ast

""" The python file we want to analyze.  Happens to be itself """
pythonfilename = 'pythonlinenumbers.py'

newcommands = []

def makenewcommand(command,
                   output):
    """ Turns the command and line number into the appropriate command.
    The signature is split onto two lines to make it more complicated.
    We have to play tricks with the trailing \, because we can't end a string
    with a single backslash. """
    return r'\newcommand{''\\'+command+'}{'+str(output)+'}\n'

class FuncLister(ast.NodeVisitor):
    def visit_FunctionDef(self, node):
        """ Recursively visit all functions, determining where each function
        starts, where its signature ends, and where the function ends.  Store
        these in the TeX variables \firstline@funcname, \sigend@funcname,
        \docend@funcname, and \lastline@funcname. """
        newname=node.name.replace('_','@') # _ isn't allowed in a TeX command
        newcommands.append(makenewcommand('firstline@'+newname,node.lineno))
        sigend = max(node.lineno,lastline(node.args))
        newcommands.append(makenewcommand('sigend@'+newname,sigend))
        docstring = ast.get_docstring(node)
        docstringlength = len(docstring.split('\n')) if docstring else -1
        newcommands.append(makenewcommand('docend@'+newname,sigend+docstringlength))
        newcommands.append(makenewcommand('lastline@'+newname,lastline(node)))
        self.generic_visit(node)

def lastline(node):
    """ Recursively find the last line of a node """
    return max( [ node.lineno if hasattr(node,'lineno') else -1 , ]
                +[lastline(child) for child in ast.iter_child_nodes(node)] )

with open(pythonfilename) as f:
    code = f.read()
FuncLister().visit(ast.parse(code))
with open('linenumbers.tex','w') as f:
    for newcommand in newcommands:
        f.write(newcommand)

Creates the file linenumbers.tex:

\newcommand{\firstline@makenewcommand}{10}
\newcommand{\sigend@makenewcommand}{11}
\newcommand{\docend@makenewcommand}{15}
\newcommand{\lastline@makenewcommand}{16}
\newcommand{\firstline@visit@FunctionDef}{19}
\newcommand{\sigend@visit@FunctionDef}{19}
\newcommand{\docend@visit@FunctionDef}{23}
\newcommand{\lastline@visit@FunctionDef}{32}
\newcommand{\firstline@lastline}{34}
\newcommand{\sigend@lastline}{34}
\newcommand{\docend@lastline}{35}
\newcommand{\lastline@lastline}{37}

Then, you can have your main TeX file input the derived TeX file, and use what it found:

\documentclass{article}

\usepackage{listings}

\immediate\write18{python3 pythonlinenumbers.py}
\makeatletter
\input{linenumbers}
\newcommand{\showfunc}[1]{%
 #1 signature:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname sigend@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}

 #1 in its entirety:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname lastline@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}}
\makeatother

\begin{document}

Here's the lastline function:

\showfunc{lastline}

\bigskip

Here's the makenewcommand function (also has a multi-line signature):

\showfunc{makenewcommand}

\end{document}

This results in

pdf output

  • Nice! To get rid of the need to run ./pythonlinenumbers.py manually everytime you update the py file, you can add \immediate\write18{python ./pythonlinenumbers.py} to the beginning of the definition of showfunc, then it will be automatically executed (as long as the user compiles with --shell-escape) when you compile the file. That should work, right? – Grimler Aug 4 '16 at 9:34
  • I actually prefer this to my own answer for robustness and not needing to close the defintion with a comment - but thought I'd provide an alternative anyway. – Chris H Aug 4 '16 at 10:23
  • @Grimler Indeed it does. Nice call. – Teepeemm Aug 4 '16 at 12:57
  • I've also just discovered that there is a pythontex package, which allows you to include executable Python code in a TeX document. This would remove the need to have a separate script file. I've not investigated further. – Teepeemm Aug 4 '16 at 15:24
2

Here's another approach for unix-like systems with sed available (tested on windows+cygwin). You need to run with --enable-write18

\documentclass{article}
\usepackage{listings}

% Here we read the occurence of "def #2" and "end def #2" from 
% file #1 into lstart and lstop respectively  
% method taken from http://tex.stackexchange.com/a/70858/28808 thanks 
% to Antal Spector-Zabusky
\newread\myinput
\newcommand{\readfunc}[2]{% first parameter is the filename, second the function name
    \leavevmode
    \immediate\write18{sed -n '/def #2/=;//q' <#1 >\jobname_start.txt}
    \immediate\write18{sed -n '/end def #2/=;//q' <#1 >\jobname_stop.txt}
    \openin\myinput=\jobname_start.txt
    \bgroup
        \endlinechar=-1
        \read\myinput to \localline
        % Since everything in the group is local, we have to explicitly make the
        % assignment global
        \global\let\lstart\localline
    \egroup
    \openin\myinput=\jobname_stop.txt
    \bgroup
        \endlinechar=-1
        \read\myinput to \localline
        \global\let\lstop\localline
    \egroup
    \lstinputlisting[firstline=\lstart,lastline=\lstop]{#1}
}

This uses GNU sed to find the "def myfunc..." line, and relies on a matching "end def myfunc" comment. I have some pretty long definitions in my python script so tend to do this anyway. Test code:

\begin{filecontents}{test.py}
#a comment
def myfunc():
    print "This is a function"
#end def myfunc

myfunc()
\end{filecontents}

\begin{document}
\subsection{Listing 1}
\readfunc{test.py}{myfunc}
\end{document}

Temporary files are used to hold the output of sed and aren't deleted. The input code could probably be optimised from what I've written.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.