Forward-inverse search using SyncTeX with the precison of a single word

Question

I would like to know if SyncTeX supports forward search and inverse search in a precision of a single word. The "default" implementation in both TeXLive and MikTeX only seems to be able to synchronize with the precision of a line.

As in Direct and reverse synchronization with SyncTEX, Jérôme Laurens mentioned:

With additional technologies such as implemented in iTEXMac2, we can even synchronize by words (see figure 1), essentially always finding an exact word correspondence between input and output.

Are there already methods to do the same thing with MikTeX or TeXLive? In fact, what I hope for is inverse search, rather than forward search. But of course, if both could be done, it is perfect.

If there is no method yet, does anyone work on this?

EDIT: See Synctex and Environments for a possible side effect (or requirement?) of a "word-by-word" SyncTeX.

Answer 1

A propos better SyncTeX'ing...

Backwards search in TeX is inherently a badly posed problem. For example, if you ctrl-click on a word in the output title of a latex document, should you be backwarded to the line containing the \maketitle command (the one that created the title) or the line containing the \title command (the one that defined the title). Both solutions are acceptable, none is universally better than the other. That being said we must accept that backwards synchronization will only be satisfying as long as tex macros are not involved too deeply. In that case, here is what can be done

• À la TeXShop : when the user ctrl+clicks somewhere on the output, take ten characters to the left and ten to the right, then try to match this sequence of 21 characters in the source. Coupling with SyncTeX will solve ambiguities
• À la iTeXMac2: when the user ctrl+clicks somewhere on the output, take the word where the click occurred, take the two words before and the two words after. Then find those 5 words in the source, in that order. Of course there are in general many occurrences, you just state that the better ones are found when the words found in the source file span over the shortest interval. Coupling with SyncTeX will solve ambiguities too.

In fact, I used only words with 3+ letters in order to be significant and avoid the high frequency of shorter words. Sometimes we do not find the whole five words, but only 4 or 3. The important thing is that ambiguities are rare. In theory, we can show all the occurences found in the backwards search, but in practice this is sometimes unreadable and a bad UI design.

Whereas iTeXMac2 works far better than TeXShop, this technology won't work with foreign language due to accented letters, except when used with utf8 encoding and the proper font. However, you can circumvent this limitation by using the Levenshtein distance or some weak variant, at a possibly significant computing price.

Anyway, all this assumes that the output is somehow readable, which means that the output viewer must be sufficiently clever. The pdf reader on OS X is really cool since a long time for that purpose, and readers on other OS's start to implement more advanced stuff little by little.

On a completely different approach, SyncTeX ing would be improved if not only line numbers but column numbers were also recorded. Unfortunately, the TeX engine only uses line numbers. I have no precise idea of the amount of work necessary to implement column numbering in TeX, but my feeling is that it is beyond my possibilities (at least in time...). But SyncTeX is already supporting column numbering, that is a first step.

Answer 2

Please forgive my ignorance -- in essence, Jérôme's paper linked above contains many answers. To sum up:

• SyncTeX is embedded deeply into TeX

• TeX does not record column information for the input file

• Thus, the best we can get is line accuracy plus added accuracy through text analyisis

With credit to percusse for the idea, how about the following approach. A TeX document is preprocessed so that each single space is converted to a newline with accurate reference to the position in the input file. The file i.tex

\begin{document}
    Lorem ipsum
\end{document}

becomes

\begin{document}%i.tex:1:1
    Lorem %i.tex:2:5
    ipsum%i.tex:2:10
\end{document}%i.tex:3:1

and is parsed equivalently by LaTeX given that the catcode of the space is not changed. (This is not exactly true, see below.) Current SyncTeX will match accurately to the line in the preprocessed file, and all that remains to do is to parse the comment in the preprocessed file.

The entire work of preprocessing and mapping could be carried out through a small wrapper around pdflatex. The filehook package by Martin Scharrer will help trigger the preprocessing of referenced files "on the fly", either through TeX code ( Use the catchfile package to preprocess an input file character by character? ) or using --shell-escape.

As egreg has mentioned, there are Macros and environments that rely on the distinction between space and newline: verbatim and lstlisting environments, \verb and \index macro, ... These must be left untouched, or else. Currently I don't see an easy way to handle this short of "expert knowledge" in the wrapper.

This is more of a rough idea, a few design issues still have to be solved. Comments are welcome.

The ultimate question here is: What is easier -- give TeX a column counter, or try to produce an equivalent document spread with one word (one character?) per line.

Answer 3

I would like to make sure first that I understand your question correctly. Are you interested in using direct and inverse search with MikTeX commonly used distribution of TeX and friends for Windows OS?

It that is problem you are trying to solve it is already solved. I am not a Windows user but I know of at least two ways that you can do inverse and direct search.

1. My favorite is installing TeXWorks IDE (consist of editor and PDF viewer) which is intensionally looks like TeXShop for MAC. You do not need to do anything in particular to do inverse search. Click on the word you are working in PDF viewer and you will be taken to source code. Direct search works similarly but you have to hold one key while clicking (I forgot which one).

2. WinEdt (TeX IDE) for Windows only could do direct and inverse search since 99. Unfortunately, WinEdt is becoming with each edition bigger and bigger bloatware which seems current trend in the modern software engineering. The good news is that I do not run Windows so I do not have even theoretical chance to use WinEdt.

There are two serious caveats of TeXWorks (at least for me).

a. First one is that by default TeXWoks uses pdflatex and I use pstricks a lot so I need dvi->ps->pdf. That one can be solved "easily".

b. The one more important which actually prevents me from using TeXWorks on my OpenBSD machine is that I have not found a way to use vi (nvi user here) key bindings in TeXWorks editor (I am looking forward for comments which are going to tell me that I am wrong). This is more serious problem as requires (unless) already exist somebody to hard code bindings into TeXWorks.

I actually have an additional question for people who might try to answer your question. I personally to not use direct and inverse search for two reasons.

i. srctex which is supposed to do synchronization doesn't work very well with large documents scattered over several files and called with \input command. Does pdfsync works better?

ii. For nvi the inverse search is tricky since every time you click on the work in Xdvi the document is open in the new instance of nvi. Direct search even doesn't work. This is not the problem that can be easily solved. My preferred solution would be if TeXWorks could do complete vi emulation not just key binding. I mean search using regular expression, shell escape command and execution of sed, awk scripts (I do not use Perl), exrc configuration. That is quite a bit to ask from an editor. The added bonus is that nvi doesn't support syntax highlighting and reliable infinite undo which TeXWorks editor supports.

Answer 4

The way I look at the problem is that with SyncTex you can get the input line number, but you do not know the word number nor the column number. You can get the total number of words in a file (and even a section) with word count. If you processed the file input line by input line you should be able to get the words per input line. I haven't looked at how word count works, but it might be possible to modify it to reset the word count after every input line. This theoretically should provide you with the word number and line number for every word. This would then leave the problem of going from word number to column number. I would think this could be left up to the editor. Instead of the editor taking a line and column number, it would have to accept a line and word number.