16

Online Judges are platforms used in competitive programming. Participants write code that solves a problem. Competitors submit their programs to the online judge, which runs all pieces of code in the grading server under the same predetermined input (if any) and environment. The judge compares the output of the programs with the correct output and determines whether the participants solved the task or not.

I was upset when I was forced to ditch my favorite language because major coding competitions did not support it. Now I am working on a project to incorporate as many languages as possible to an online judge, focusing on unpopular languages. Recently I successfully integrated *sh and powershell. While searching for new candidates, the TeX family came to mind. It would be a fabulous addition to the group.

My plan was to compile, collate all PDF pages to a long image, OCR it then compare it with the right output. That is, pdflatex | pdftocairo | tesseract | diff. However I was met with numerous challenges. Since TeX writes to PDF which has a length limit, problems that require users to output a long line of text cannot be judged properly.

For example, consider a basic problem that asks contestants to simply print the numbers 1 to 100 in a line, separated by one space. Using C, one could do this:

$ cat ./example.c
#include <stdio.h>

int main() {
    for (int i = 1; i <= 100; i++) {
        printf("%d ", i);
    }
    return 0;
}

The code would be judged by this process:

$ gcc ./example.c -o ./example.o
$ ./example.o
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

This would be diffed and the online judge would rule it correct. However in LaTeX the output would inevitably span on multiple lines, resulting in a different output when ocr'd. Formatting matters in competitive programming and this cannot be deemed correct. I can't modify tesseract params to assume all output is in one line, since that would break any question that requires more than two lines of output. Giving exemptions to TeX would be unfair for other languages.

$ cat ./example.tex
\documentclass{article}
\usepackage{pgffor}
\pagenumbering{gobble}
\begin{document}
\foreach \i in {1,...,100} {%
    \i\space%
}%
\end{document}
$ pdflatex ./example.tex
$ pdftocairo ./example.pdf -jpeg -singlefile
$ tesseract -c debug_file=/dev/null ./example.jpg -
123456789 1011 12 13 1415 1617 18 19 20 21 22 23 24 25 26 27 28
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

Not only does the output span over multiple lines, numbers 1 to 9 were erroneously combined into a single string by tesseract. Would there be a feasible way to adjust this procedure so that there would be a reliable automated grading system for TeX akin to other established languages that would not give it a disadvantage nor an advantage?

If this is not viable the only other way would be to make TeX output to stdout or a plaintext file like all others. Can TeX be instructed to do this?

9
  • 2
    Just an idea: instead of actually looking at the pdf, maybe you could use a similar approach as the regression testing of l3build? Aug 14, 2023 at 19:52
  • 5
    Why don't you just print the messages in the log? That one is length-limited as well, but you could circumvent that by additional mark up in your output that you then parse (e.g., ignoring all new-lines of the output block, and printing an explicit \n to the terminal). Since you need additional parsing anyway, I'd argue that this is the easier route. In LaTeX you could use \typeout for this (adds a new line after the content), or the TeX primitive \message.
    – Skillmon
    Aug 14, 2023 at 19:54
  • 7
    why typeset the result ? just \write to a file Aug 14, 2023 at 19:56
  • 4
    And else: Yes, TeX supports writing to an arbitrary file name, \openout, \write, and \closeout are the relevant primitives.
    – Skillmon
    Aug 14, 2023 at 19:57
  • 3
    And if you want to have a slightly easier life you could use the LaTeX3 programming language described in texdoc expl3 and texdoc interface3 (those are commands in the command line).
    – Skillmon
    Aug 14, 2023 at 19:58

4 Answers 4

21
\documentclass{article}
\usepackage{pgffor}
\pagenumbering{gobble}
\newwrite\res
\immediate\openout\res=\jobname.txt
\begin{document}
\def\x{}
\foreach \i in {1,...,100} {%
    \xdef\x{\x\i\space}
}%
\immediate\write\res{\x}
\end{document}

leaves a 1-line text file .txt extension containing

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 
4
  • Why the \begin{document}? Just \stop after \immediate\write\res{\x} and leave out the unnecessary stuff :P
    – Skillmon
    Aug 14, 2023 at 20:06
  • 1
    @Skillmon sure, and as you say it would be simpler with L3 but just minimised the differences to highlight the \write Aug 14, 2023 at 20:08
  • 1
    @Skillmon if you were egreg you'd have posted an l3 answer by now to try to steal a tick Aug 14, 2023 at 20:10
  • 1
    Good idea, I'll provide an L3 answer :P
    – Skillmon
    Aug 14, 2023 at 20:13
16

The following creates a file with the same name as your .tex-file but with the extension .txt that will contain all numbers from 1 to 100 followed by a space.

This example uses the L3 language, which adds a coherent programming language to TeX (still by macro expansion).

\ExplSyntaxOn

\iow_open:Nn \g_tmpa_iow { \jobname.txt }
\tl_clear:N \l_tmpa_tl
\int_step_inline:nn { 100 } { \tl_put_right:Nn \l_tmpa_tl { #1~ } }
\iow_now:Nx \g_tmpa_iow { \l_tmpa_tl }
\iow_close:N \g_tmpa_iow

\stop

Results in

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 

Another possibility instead of using many calls to \tl_put_right:Nn is using the \tl_build_... functions, which provide much better performance for piece-wise build token lists.

\ExplSyntaxOn

\iow_open:Nn \g_tmpa_iow { \jobname.txt }
\tl_build_begin:N \l_tmpa_tl
\int_step_inline:nn { 100 } { \tl_build_put_right:Nn \l_tmpa_tl { #1~ } }
\tl_build_end:N \l_tmpa_tl
\iow_now:Nx \g_tmpa_iow { \l_tmpa_tl }
\iow_close:N \g_tmpa_iow

\stop

A slightly shorter code variant using an expandable function to map over the integers from 1 to 100 and an expandable macro adding a space (TeX can be weird when it comes to stuff being expandable or not, so this might be the harder to use variant in writing to a file if you have more complicated things to be done):

\ExplSyntaxOn

\iow_open:Nn \g_tmpa_iow { \jobname.txt }
\cs_new:Nn \my_addspace:n { #1~ }
\iow_now:Nx \g_tmpa_iow { \int_step_function:nN { 100 } \my_addspace:n }
\iow_close:N \g_tmpa_iow

\stop

And a fully expandable variant with even better performance:

\ExplSyntaxOn

\iow_open:Nn \g_tmpa_iow { \jobname.txt }
\cs_new:Npn \my_spaceloop:nN #1#2
  { #1~ \exp_args:Ne #2 { \int_eval:n { #1 + 1 } } }
\iow_now:Nx \g_tmpa_iow
  {
    \exp_args:NNNo \exp_last_unbraced:NNo \my_spaceloop:nN 1
      { \prg_replicate:nn {99} \my_spaceloop:nN }
    \use_none:n
  }
\iow_close:N \g_tmpa_iow

\stop
4
  • +1 although I'd have expected you of all people to do an expandable version inside the write Aug 14, 2023 at 20:30
  • @DavidCarlisle was considering it, but thought this is the conceptually easier way for someone who is not familiar with L3 yet.
    – Skillmon
    Aug 14, 2023 at 20:58
  • For the \l_tmpa_tl version, you might consider the \tl_build commands, which should be more efficient than repeating \tl_put_right:Nn. With \benchmark:n I get about 1200 ops with your method, about 900 with the \tl_build method. They become, respectively, almost 100000 and about 10000 when the numbers from 1 to 1000 are processed. Of course, the \int_step_function:nN approach is much more efficient.
    – egreg
    Aug 15, 2023 at 21:14
  • @egreg agreed, \tl_build... is much more efficient for bigger token lists, but the idea was to show simple approaches (which is why I didn't even include the \int_step_function:nN at first). I might add a \tl_build... version later today... :)
    – Skillmon
    Aug 16, 2023 at 6:26
8

The other answers change the input document, one a bit more radical than the other. This may not be desired for a programming competition, where you would want to judge somewhat 'natural' code.

Instead of compiling to pdf you could also compile to html using tex4ht. This allows the input to be the same, and produces a html file containing plain text which may be easier to diff than PDF+OCR over multiple lines etc.

Basic example:

example.tex

\documentclass{article}
\usepackage{pgffor}
\pagenumbering{gobble}
\begin{document}
\foreach \i in {1,...,100} {%
    \i\space%
}%
\end{document}
make4ht example.tex

Resulting example.html:

<!DOCTYPE html> 
<html lang='en-US' xml:lang='en-US'> 
<head><title></title> 
<meta charset='utf-8' /> 
<meta content='TeX4ht (https://tug.org/tex4ht/)' name='generator' /> 
<meta content='width=device-width,initial-scale=1' name='viewport' /> 
<link href='example.css' rel='stylesheet' type='text/css' /> 
<meta content='example.tex' name='src' /> 
</head><body>
<!-- l. 7 --><p class='noindent'>1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
100
</p>
    
</body> 
</html>

Of course executing a for loop to print a list of numbers is something that is not very TeX-like, and it would therefore be a poor choice for a programming competition. You could also assign the participants a math exercise. For this you can choose between an image format in the output, which is not suitable for automatic judging, or MathML, or MathJax.

Examples:

\documentclass{article}
\usepackage{pgffor}
\pagenumbering{gobble}
\begin{document}
\foreach \i in {1,...,100} {%
    \i\space%
}%

$c = \sqrt{a^2+b^2}$
\end{document}

Terminal command for MathML:

make4ht example.tex "mathml"

Resulting html (snippet, lightly formatted for readability):

<!-- l. 9 --><p class='indent'>   <!-- l. 9 -->
<math display='inline' xmlns='http://www.w3.org/1998/Math/MathML'>
<mrow>
   <mi>c</mi> <mo class='MathClass-rel'>=</mo>
   <msqrt>
      <mrow><msup><mrow><mi>a</mi></mrow>
      <mrow><mn>2</mn> </mrow> </msup> 
      <mo class='MathClass-bin'>+</mo> <msup><mrow><mi>b</mi></mrow>
      <mrow><mn>2</mn></mrow></msup></mrow>
   </msqrt>
</mrow>
</math>
</p>

Terminal command for MathJax:

make4ht example.tex "mathjax"

Html snippet:

<!-- l. 9 --><p class='indent'>   \(c = \sqrt {a^2+b^2}\)
</p>

If you want to make parsing/diff-ing the file a bit easier then you can introduce one or more environments in the LaTeX file and assign a html class attribute to the part of the output that corresponds to that exercise. This requires a separate configuration file for TeX4ht.

mycompetition.cfg

\Preamble{xhtml}
\ConfigureEnv{assignment}
    {\HCode {<div class="assignment">}} 
    {\HCode {</div>}} {} {}
\begin{document}
\EndPreamble

example.tex

\documentclass{article}
\newenvironment{assignment}{}{}
\usepackage{pgffor}
\pagenumbering{gobble}
\begin{document}
\begin{assignment}
\foreach \i in {1,...,100} {%
    \i\space%
}%
\end{assignment}

\begin{assignment}
$c = \sqrt{a^2+b^2}$
\end{assignment}
\end{document}

Terminal command:

make4ht -c mycompetition.cfg example.tex "mathjax"

Html output snippet:

<!-- l. 6 --><p class='noindent'><div class='assignment'> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
100
</div>
</p><!-- l. 12 --><p class='noindent'><div class='assignment'> \(c = \sqrt {a^2+b^2}\)
</div>
</p>

You could try to extend this to include for example an assignment counter as argument to the environment, see TeX4ht : How to configure new environment with arguments? for some pointers on how to approach this.

1

EDIT: The following approach doesn't work for the intended purpose, as Pandoc will not really evaluate all of tex (unless I am missing something) and thus e.g. the foreach in the example does not render the numbers via Pandoc...

Unless the goal is to test fine-grained skills in Latex formatting, I think the best output format is Markdown - you can use Pandoc to do the conversion (see Example 5 at https://pandoc.org/demos.html). The desirable properties of markdown output is that it is

  1. almost plain text, so easy to diff
  2. somewhat robust to different latex coding styles - i.e. where mild changes to Latex source would create mildly different PDF or add extra <div> tags to HTML output, they could still result in exactly identical Markdown output

You'd probably want to set --wrap=preserve to avoid having the output wrapped to 80 char width (which is the default)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .