19

What is the recommended way of writing a package in the .dtx format? Do package authors usually just write the .dtx first, or use some kind of tool to create a .dtx from the .sty files? I know both approaches are valid, but what's the best way to get started? I'm currently trying to make sense of the available documentation, but like LaTeX in general this feels like it has a learning curve not unlike a brick wall.

Can LaTeX load .dtx files as a substitute for .sty files, or must they be preprocessed first?

Prior packages that I've written were just collections of \usepackage and various convenience macros, but now I'm trying to write something more useful that I'd like to distribute at some point, and contribute back to the community.

  • Do you know about makedtx? Or l3build? – TeXnician Nov 18 '17 at 7:34
  • Not really. I'm using this as an exercise to learn how to make a LaTeX3 package. – Robbie Nov 18 '17 at 7:39
  • The .dtx format (and associated doctex mode of Emacs/AUCTeX) is about ok for code comments but is a pain for a full-fledged user manual therein. In the past, I used special set-up so the user manual would stand "uncommented" in .dtx. But for bigger packages, the .dtx is but a Makefile target. In particular I have private comments in my sources, which the build script removes. Also it will test the .sty extraction from the produced .dtx and run tests with them, even if the original .sty sources were already tested. make, sed, python... is easier for me – user4686 Nov 22 '17 at 9:37
  • ...but naturally l3build is probably way to go in your context. – user4686 Nov 22 '17 at 9:37
  • I've got l3build working just fine, and I've taken the effort to document my code as I go. The only really thoroughly-documented LaTeX3 sources I've been able to find from casually browsing my TeXLive tree is siunitx, which has helped a lot in trying to understand how LaTeX3 works in practice because some of the examples in the available documentation so far are a bit opaque (hence my recent poorly-worded questions here). The only thing I haven't gotten l3build to do so far is pick up the version and date tags. – Robbie Nov 22 '17 at 9:57
12

Disclaimer: I am a package developer, but solely for internal use (me/my institution; 9 packages and 3 classes so far). I have not uploaded a package to CTAN yet. Hence this is not the advice a more experienced LaTeX developer might give.

I usually structure the development of my packages (which are dtx files at the end, using l3build as working environment) into three stages:

Stage 1: Getting the concept right

This stage involves trial and error. I set up a test document with the syntax I want and make "dummy commands" for all undefined control sequences, which effectively do nothing at all.

Then I'm setting up a basic package as a .sty file (using expl3 syntax). Now I'm trying to implement the essential macros which are needed as well as package options. For each macro implemented I remove my dummy macro.

This step lasts until I think that the basic functionality of my package is implemented. While implementing you are easily able to test.

Stage 2: Documenting your progress

In this step I'm transferring the code into dtx format (using the l3doc functionality). This step is very tedious, but important. Basically you add the comment structure to your code. At this point I also make the package a git project for version control.

Do not forget to write down something about the basic concept of your package and a simple roadmap, especially if you know you will have pauses in development.

While documenting I utilize l3build to build the documentation for me. Usually I also include my package in my dtx documentation (which may contain examples), so severe issues will be recognized very fast. l3build lets you also create test files, which are a good option for more complicated packages.1

Stage 3: Further development, tests and releases

Your dtx file is ready for additional functionality and updates of existing macros. I'm altering the dtx file directly (git is my insurance if something goes wrong). As l3build also creates the package with every run, if you tell it so2, you may also just copy the .sty file and test some modifications with that beforehand.

As I include the package within the dtx documentation and usually write up examples for the functions of my package I will also notice some issues just while extending the package.

If you think you are at a point the functionality works, turn your dtx file into a full documentation (for the user) and think of some test cases (what can the user do wrong). Test the package. I usually take my test file from step 1 as a starter.

Then you can use l3build to cmdcheck your dtx and make a ctan build. You may create a git tag at this point (if you're using git).


1 It is slightly harder to do this for classes as you probably won't write your documentation with your class, but you may want to include some example file into the documentation which is compiled every doc build just for testing.
2 You may basically include the .ins file into the .dtx file and get the package created at every run. There are examples out there how to do it.

  • Is there a manual for how to document code, i.e. for the commands that will be read to generate documentation? I'm sure there is, but I don't know what to search for with texdoc. – Robbie Nov 19 '17 at 6:48
  • @Robbie You mean dtx format? How to use it? Look at the documentation of l3doc and doc. – TeXnician Nov 19 '17 at 8:43
  • I think you should start your answer with stating that you use l3build as workflow environment. Something like test driven development is so much easier if you can use a machinery already ready for you to simply plug tests in etc – Frank Mittelbach Nov 22 '17 at 8:36
10

The first thing to remember is that you do not have to use the .dtx format to include comments in package code: you can just use a .sty. Assuming you do want to use .dtx to allow typesetting the code, etc., then you still may be able to use the .dtx directly. That is true if there is only a single 'target' in the source file: just comments and the 'live' code. However, most .dtx files include the 'driver' (to allow typesetting the code), and so effectively at least two targets. At this stage you have to extract the .sty to be able to use it with LaTeX.

If we take this common situation, you can test by having a simple .tex file in the same place as the .dtx and using shell escape to enable a first line such as

\immediate\write18{tex mypkg.dtx}
\documentclass{article}
\usepackage{mypkg}
% Tests

That of course has a non-zero security implication and tends to be a bit 'fragile': ideally you want a sandbox for testing that keeps extracted code separate from everything else.

The obvious approach is to script extraction, perhaps installation in the local tree and testing. There are several scripts which can do that, but unsurprisingly I'd point to l3build from the LaTeX3 Project. This is Lua-based script which allows fully automated testing of packaged code: depending on what you want, you might just unpack and test with a simple .tex file, or might create one or more automated test input/output sets (.lvt/.tlg files).

In terms of creating .dtx files, there are scripts to 'fill in the blanks' but in the end the point of the format is to have useful comments/description both for documentation and in the code. That can only be done by you actually writing the material!

  • I want to learn the agreed-upon "best" practises, which means creating self-documenting code and structured packages. The idea is to avoid future work by doing it right the first time. Following the advice of an earlier comment, I've got a basic l3build script running, which suffices; it's not a big deal to run ./build.lua install and then lualatex test.tex. At the moment the package I've got is so basic that testing mostly consists of looking at the output to see whether it does what I want, though I do intend to create some testing cases at some point in the future. – Robbie Nov 19 '17 at 3:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.