11

I'm sorry, if this question isn't appropriate here, since it asks (at least partly) for opinions. However, there exists a "best-practices" tag, and I believe, the answer could be of public interest.

Background

In our workgroup I maintain a small bundle consisting of a class, several packages, and perl scripts, in total about 4000 LoC. The bundle is subject of continuous development. Currently, I prepare a new major version, that includes a few new files.

Until now, I've distributed the code as collection of files in a directory or archive. If a group member wanted to install the bundle, she/he had to copy every file to its appropriate place.

With the new revision, I want to switch to a more "TeXish" development: literate programming and bundle installation with .dtx and .ins files.

Problem

While I believe that I understood the format of the .dtx file and how docstrip works, I'm still puzzled about the development workflow.

As any development, I follow a development cycle:

edit - compile - testing - commit (if the tests succeed).

After few of such cycles, a bug is eliminated or a new feature is established (and a new version can be shipped). However, if the source consist of one or several .dtx files, the cycle would include at least an additional installation step. Considering that the edit changes are usually rather small, an additional compilation/installation seems to be rather inefficient.

I've considered several options, but I find none of them very appealing :

  • Using one big dtx file. Disadvantage: editing is quite unhandy and there is the additional compilation/installation step for each test step.
  • Using one dtx file for each package and the class. Disadvantage: now I have even more installation steps.
  • Using sty2dtx, not in every cycle, but if a feature is ready. Disadvantage: Several, e.g. sty2dtx can't handle LaTeX3 command definitions.
  • Writing a kind of diver program that compiles a .dtx file from the source files, while the source file stay valid TeX/Perl files with special comments. Disadvantage: I'm not sure whether my TeX foo is sufficient. In addition: I have the feeling that, if this would be a good idea, somebody would have done this already.

Question

What is a good TeXish workflow that support continuous development and results in an installation bundle? Did I miss an option?

1
  • 3
    the latex team (either as team or individuals, or other hats such as ho-tex support group) maintain hundreds of dtx files, and for most have test suites, and/or continuous integration github actions. The main tool is l3build but you can look at the full build actions at the latex3 repositories Oct 25, 2022 at 16:20

1 Answer 1

10

Whilst one can handle .dtx sources 'by hand', I think it's best to think of code set up in this way like other source-program relationship: there is a build process. This is really independent of whether you use a single .dtx or several (I would favour the latter approach for all but the smallest projects).

With your source in .dtx format, extracting to usable code, executing tests, installing, etc., is best delegated to a build tool. You can of course write this yourself or use a system such as make. However, the LaTeX team have developed l3build precisely because there are particular aspects to building TeX code that are different from other languages. This includes the need to write a 'matching' testing environment. With l3build, checking a modified piece of code is as simple as

l3build check

if you have tests set up, or if you want to install and run locally

l3build install

followed by your manual testing, and probably

l3build uninstall

once you are done.


Note that whilst l3build has l3 in the name, that's because it was written as a tool to support installation of l3... files. However, it is intended to be a generic tool: it works with a range of code structures, and can happily work with LaTeX2e, plain or even (to some extent) ConTeXt code. We have over time added a range of features to support different development methods, including custom installation locations, non-standard test support, etc.

2
  • Thank you for your answer. I'm not deep in l3, more on the interface to e2: (formerly) xparse, key-value, strings, etc. It was my understanding, that l3build is more or less for the internal development within the l3 project only. But now, I definitively will have a look at it.
    – Matthias
    Oct 26, 2022 at 10:43
  • @Matthias We wrote l3build originally for LaTeX kernel development, but generalised it some years ago now: it's used by lots of 'pure 2e' packages (for example testing pgf, biblatex, beamer), and by us for generic code (unicode-data, for example).
    – Joseph Wright
    Oct 26, 2022 at 11:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .