Best practice for dtx workflow

Question

I'm sorry, if this question isn't appropriate here, since it asks (at least partly) for opinions. However, there exists a "best-practices" tag, and I believe, the answer could be of public interest.

Background

In our workgroup I maintain a small bundle consisting of a class, several packages, and perl scripts, in total about 4000 LoC. The bundle is subject of continuous development. Currently, I prepare a new major version, that includes a few new files.

Until now, I've distributed the code as collection of files in a directory or archive. If a group member wanted to install the bundle, she/he had to copy every file to its appropriate place.

With the new revision, I want to switch to a more "TeXish" development: literate programming and bundle installation with .dtx and .ins files.

Problem

While I believe that I understood the format of the .dtx file and how docstrip works, I'm still puzzled about the development workflow.

As any development, I follow a development cycle:

edit - compile - testing - commit (if the tests succeed).

After few of such cycles, a bug is eliminated or a new feature is established (and a new version can be shipped). However, if the source consist of one or several .dtx files, the cycle would include at least an additional installation step. Considering that the edit changes are usually rather small, an additional compilation/installation seems to be rather inefficient.

I've considered several options, but I find none of them very appealing :

• Using one big dtx file. Disadvantage: editing is quite unhandy and there is the additional compilation/installation step for each test step.
• Using one dtx file for each package and the class. Disadvantage: now I have even more installation steps.
• Using sty2dtx, not in every cycle, but if a feature is ready. Disadvantage: Several, e.g. sty2dtx can't handle LaTeX3 command definitions.
• Writing a kind of diver program that compiles a .dtx file from the source files, while the source file stay valid TeX/Perl files with special comments. Disadvantage: I'm not sure whether my TeX foo is sufficient. In addition: I have the feeling that, if this would be a good idea, somebody would have done this already.

Question

What is a good TeXish workflow that support continuous development and results in an installation bundle? Did I miss an option?

Answer 1

Whilst one can handle .dtx sources 'by hand', I think it's best to think of code set up in this way like other source-program relationship: there is a build process. This is really independent of whether you use a single .dtx or several (I would favour the latter approach for all but the smallest projects).

With your source in .dtx format, extracting to usable code, executing tests, installing, etc., is best delegated to a build tool. You can of course write this yourself or use a system such as make. However, the LaTeX team have developed l3build precisely because there are particular aspects to building TeX code that are different from other languages. This includes the need to write a 'matching' testing environment. With l3build, checking a modified piece of code is as simple as

l3build check

if you have tests set up, or if you want to install and run locally

l3build install

followed by your manual testing, and probably

l3build uninstall

once you are done.

---

Note that whilst l3build has l3 in the name, that's because it was written as a tool to support installation of l3... files. However, it is intended to be a generic tool: it works with a range of code structures, and can happily work with LaTeX2e, plain or even (to some extent) ConTeXt code. We have over time added a range of features to support different development methods, including custom installation locations, non-standard test support, etc.