How to query if a file has changed (a \IfFileHasChanged conditional)?

Question

We know \IfFileExists{<file>}{<yes>}{<no>}.

How can I create a command \IfFileHasChanged{<file>}{<yes>}{<no>}
for a file which has changed (and does exist!).

I mean it this way:

• If I have myfile.txt with This is my file. it should be written in the first (with filecontents).

• If I change myfile.txt to Now this is my file... it should be new-written.

• else: do not write new.

Maybe this has something to do with the "timestamp".

\begin{filecontents*}[overwrite]{myfile.txt}
This is my file.
\end{filecontents*}

\documentclass{article}
\begin{document}
\section{IfFileExists}
\IfFileExists{myfile.txt}{\input{myfile.txt}}{Does not exist!}

\section{IfFileHasChanged}
\end{document}

Answer 1

Checking if a file has changed, as Ulrike says, requires to compare it against some previous state of the file.

Here's an implementation using roughly the same method as in Rmano's answer of storing the MD5 sum of the file in the .aux file, and also storing one MD5 per file in a property list, so that you can have multiple files.

\IfFileChangedTF checks if a file has changed relative to a previously known state by querying the MD5 of the file, and comparing it to the previous known value, and returns <false> or <true> accordingly. The first time a file is checked, no MD5 is known, so the conditional returns <true> as well.

The conditional has side effects, however. When the <true> branch is taken, the property list is updated with the new MD5 sum, which means that two consecutive runs of the command may yield a different result. This is especially true in the first run: when you first execute \IfFileChangedTF{some-file}, the command doesn't know some-file, so it stores the MD5 sum and returns <true>. The next time you run \IfFileChangedTF{some-file}, however, the file is already known, so if it didn't change the conditional returns <false>.

Also, since the state is stored in the .aux, before \begin{document} the first time you call \IfFileChangedTF for a file will always return <true>.

Running your sample document once produces (one “Didn't change”):

And running it again produces (two “Didn't change”):

\begin{filecontents*}[overwrite]{myfile.txt}
This is my file.
\end{filecontents*}

\documentclass{article}
\usepackage{xparse}
\pagestyle{empty}
\ExplSyntaxOn
\prop_new:N \g__cis_file_mdfive_prop
\tl_new:N \l__cis_tmpa_str
\tl_new:N \l__cis_tmpb_str
\NewDocumentCommand \IfFileChangedTF { m +m +m }
  { \cis_file_if_changed:nTF {#1} {#2} {#3} }
\prg_new_protected_conditional:Npnn \cis_file_if_changed:n #1 { T, F, TF }
  {
    \file_if_exist:nTF {#1}
      {
        \file_get_mdfive_hash:nN {#1} \l__cis_tmpb_str
        \prop_get:NnNTF \g__cis_file_mdfive_prop {#1} \l__cis_tmpa_str
          {
            \str_if_eq:NNTF \l__cis_tmpa_str \l__cis_tmpb_str
              { \prg_return_false: }
              {
                \__cis_mdfive_update:nN {#1} \l__cis_tmpb_str
                \prg_return_true:
              }
          }
          {
            \__cis_mdfive_update:nN {#1} \l__cis_tmpb_str
            \prg_return_true:
          }
      }
      { \msg_error:nnn { cis } { file-not-found } {#1} }
  }
\makeatletter
\cs_new_protected:Npn \cis@mdfive@update #1 #2
  { \prop_gput:Nnx \g__cis_file_mdfive_prop {#1} {#2} }
\cs_new_protected:Npn \cis@mdfive@save #1 #2
  { \iow_now:Nx \@auxout { \exp_not:N \cis@mdfive@update {#1} {#2} } }
\cs_new_protected:Npn \__cis_mdfive_update:nN #1 #2
  { \cis@mdfive@update {#1} {#2} }
\AtEndDocument
  {
    \prop_map_inline:Nn \g__cis_file_mdfive_prop
      { \cis@mdfive@save {#1} {#2} }
  }
\makeatother
\msg_new:nnn { cis } { file-not-found }
  { File~'#1'~not~found. }
\ExplSyntaxOff

\begin{document}
\section{IfFileExists}
\IfFileExists{myfile.txt}{\input{myfile.txt}}{Does not exist!}

\section{IfFileChangedTF}
\IfFileChangedTF{myfile.txt}{\input{myfile.txt}}{Didn't change!}

\IfFileChangedTF{myfile.txt}{\input{myfile.txt}}{Didn't change!}
\end{document}

Answer 2

I suppose that the situation is the following. We have a LaTeX file, call it filechanged.tex, that is run with pdflatex filechanged. In the same directory, I have another file, myfile.txt.

When I run pdflatex filechanged, I want to do different processing if myfile.txt has changed since the previous run or not.

My solution:

1. It is based on @Skillmon suggestion here and @egreg code here;

2. it will write on the .aux file (which is automatically read at the start of the run) the value of the MD5 checksum of the file myfile.txt;

3. it will check if it has changed since last run and do different things if yes or not;

4. it needs a quite recent LaTeX distribution (no idea when the \file_get_mdfive_hash:nN was introduced)

5. it is my first LaTeX3 program, so it's probably full of errors.

Seems to work here... So you have two files, the first one is mytext.txt

something here

and then the main file:

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
    \cs_new:Npn \dobold #1
    {
        \textbf{#1}
    }
    \str_new:N \g_myfile_name
    \str_gset:Nn \g_myfile_name {myfile.txt}
    \str_new:N \g_myfile_old_mdfive
    \str_new:N \l_myfile_mdfive
    \cs_new:Npn \getmdfive
    {
        \file_get_mdfive_hash:nN {\str_use:N \g_myfile_name} \l_myfile_mdfive
        \str_use:N \l_myfile_mdfive
    }
    \cs_new:Npn \getoldmdfive
    {
        \str_use:N \g_myfile_old_mdfive
    }
    \cs_new:Npn \IfMyfileChanged #1 #2
    {
        \str_if_eq:NNTF \g_myfile_old_mdfive \l_myfile_mdfive {#2} {#1}
    }
    \AtEndDocument
    {
        \iow_now:cx { @auxout }
        {
            \token_to_str:N \ExplSyntaxOn
            ^^J
            \str_gset:Nn \token_to_str:N \g_myfile_old_mdfive {\str_use:N \l_myfile_mdfive}
            ^^J
            \token_to_str:N \ExplSyntaxOff
        }
    }
\ExplSyntaxOff

\begin{document}

MD5 was \getoldmdfive

MD5 is now \getmdfive

Changed? \IfMyfileChanged{Yes, it has changed}{No, it's the same as before}

\end{document}

Answer 3

I can offer an environment DifferentFileContents which has the same syntax as the filecontents*-environment.

The content of the DifferentFileContents-environment will be compared to the content of the file specified.

In case the contents differ or the file specified does not exist, the file will be destroyed and rewritten/will be created anew with the content of the environment.

This may be useful for reducing the amount of write-operations on Solid-State-Drives.

Internally the filecontents*-environment is used for this.

An optional argument, if present with the call to the DifferentFileContents-environment, is handed over to the filecontents*-environment.

If the filecontents*-environment (LaTeX 2ε-release v1.3c, 2019/09/11 and newer) does process optional arguments, things should be alright.

If the filecontents*-environment (LaTeX 2ε-release older than v1.3c, 2019/09/11) does not process optional arguments, the opening square bracket [ of the optional argument will be taken for the name of the file to create and an attempt at creating such a file will take place. The remainder of the optional argument and the non-optional filename-argument will cause all kinds of error messages.

This means:

If you use a LaTeX 2ε-release where the filecontents*-environment processes an optional argument (LaTeX 2ε-release v1.3c, 2019/09/11 and newer), then the DifferentFileContents-environment can also process an optional argument.

If you use a LaTeX 2ε-release where the filecontents*-environment does not processes an optional argument (LaTeX 2ε-release older than v1.3c, 2019/09/11), then the DifferentFileContents-environment also cannot process an optional argument.

---

The example below creates a file myfile.txt.

Hereby an already existing file myfile.txt may get destroyed/overridden.

After compiling the log-file and the terminal will contain messages informing the user whether content of environment and content of file were the same and therefore the file was not created anew / whether content of environment and content of file were not the same and therefore the file was created anew.

\documentclass{article}
\usepackage{filecontents}

\makeatletter
\newcommand\PassFirstToSecond[2]{#2{#1}}%
\newcommand\Exchange[2]{#2#1}%
%%-----------------------------------------------------------------------------
%% Stringify the first token of the second argument:
%%.............................................................................
\newcommand\UD@StringifySecond[2]{%
  \expandafter\PassFirstToSecond\expandafter{\string#2}{#1}%%
}%
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\newcommand\UD@CheckWhetherNull[1]{%
  \romannumeral0\expandafter\@secondoftwo\string{\expandafter
  \@secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \@secondoftwo\string}\expandafter\@firstoftwo\expandafter{\expandafter
  \@secondoftwo\string}\@firstoftwo\expandafter{} \@secondoftwo}%
  {\@firstoftwo\expandafter{} \@firstoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%%  \romannumeral0\UD@ExtractFirstArgLoop{ABCDE\UD@SelDOm} yields  {A}
%%
%%  \romannumeral0\UD@ExtractFirstArgLoop{{AB}CDE\UD@SelDOm} yields  {AB}
%%.............................................................................
\@ifdefinable\UD@RemoveTillUD@SelDOm{%
  \long\def\UD@RemoveTillUD@SelDOm#1#2\UD@SelDOm{{#1}}%
}%
\newcommand\UD@ExtractFirstArgLoop[1]{%
  \expandafter\UD@CheckWhetherNull\expandafter{\@firstoftwo{}#1}%
  { #1}%
  {\expandafter\UD@ExtractFirstArgLoop\expandafter{\UD@RemoveTillUD@SelDOm#1}}%
}%
%%-----------------------------------------------------------------------------
\newcommand\DifferentFilecontentsVerbatimcatcodes{%
  \let\do=\@makeother
  \dospecials
  \catcode\endlinechar=12 %
  \catcode`\^^I=12 %
}%
\newcommand\DifferentFilecontents{%
  \begingroup
  \@ifnextchar[\DifferentFilecontents@opt\DifferentFilecontents@noopt
}%
\newcommand\DifferentFilecontents@opt[2][]{%
  \DifferentFilecontentsVerbatimcatcodes
  \expandafter\@DifferentFilecontents\expandafter{\the\inputlineno}{[{#1}]}{#2}%
}%
\newcommand\DifferentFilecontents@noopt[1]{%
  \DifferentFilecontentsVerbatimcatcodes
  \expandafter\@DifferentFilecontents\expandafter{\the\inputlineno}{}{#1}%
}%
\newcommand\DifferentFilecontentsEqualMessage[2]{%
  \GenericWarning{%
    \space\space\space\@spaces\@spaces\@spaces\@spaces
  }{%
    LaTeX Information: %
    Seems the content of the existing file `#1'\MessageBreak
    equals the content of environment DifferentFilecontents\MessageBreak%
    on input lines #2 - \the\inputlineno.\MessageBreak
    The file will not be overwritten/will not be created anew\@gobble
  }%
}%
\newcommand\DifferentFilecontentsDifferentMessage[2]{%
  \GenericWarning{%
    \space\space\space\@spaces\@spaces\@spaces\@spaces
  }{%
    LaTeX Information: %
    Seems either the file `#1' does not exist\MessageBreak
    or its content does not equal the content of environment\MessageBreak
    DifferentFilecontents on input lines #2 - \the\inputlineno.\MessageBreak
    The file will be overwritten/created anew\@gobble%
  }%
}%

\newread\filecompareread

\begingroup
\catcode\endlinechar=12\relax%
\edef\delimitersbehind{\@backslashchar end\string{DifferentFilecontents\string}}%
\edef\filecontentsbegin{\@backslashchar begin\string{filecontents*\string}}%
\edef\filecontentsend{\@backslashchar end\string{filecontents*\string}}%
\newcommand\@DifferentFilecontents[4]{%
  \endgroup%
  \@ifdefinable\@DifferentFilecontents{%
    % ##1 - input-line-number of begin of environment
    % ##2 - optional arguments
    % ##3 - file name
    % ##4 - environment body
    % #1 = \end{readenvironmentbody}
    % #2 = ^^M
    % #3 = \begin{filecontents*}
    % #4 = \end{filecontents*}%
    \long\def\@DifferentFilecontents##1##2##3##4#2#1{%
      \immediate\openin\filecompareread=##3\relax%
      %\message{Environment-Body: (\detokenize{|##4|})}%
      \UD@CheckWhetherNull{##4}{%
        \DifferentFilecontentsCompareLoop{^^M}%
      }{%
        \expandafter\DifferentFilecontentsCompareLoop\expandafter{\@gobble##4^^M^^M}%
      }%
      {##2}{##3}{##4}{}{##1}%
      {%
         \immediate\closein\filecompareread\relax%
         \endgroup%
      }%
      \end{DifferentFilecontents}%
    }%
  }%
  \newcommand\DifferentFilecontentsCompareLoop[6]{%
    % ##1 - remainder of environment body
    % ##2 - optional arguments
    % ##3 - file name
    % ##4 - entire environment body
    % ##5 - remainder of current line of file
    % ##6 - input-line-number of begin of environment
    % #1 = \end{readenvironmentbody}
    % #2 = ^^M
    % #3 = \begin{filecontents*}
    % #4 = \end{filecontents*}%
    \UD@CheckWhetherNull{##5}{%
      %\message{1}%
      \ifeof\filecompareread\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi%
      {%
        %\message{1-1}%
        \UD@CheckWhetherNull{##1}{%
          %\message{1-1-1}%
          \Exchange{%
            \DifferentFilecontentsEqualMessage{##3}{##6}%
          }%
        }{%
          %\message{1-1-2}%
          \UD@CheckWhetherNull{##4}{%
            %\message{1-1-2-1}%
            %\message{Writing:(#3{##3}#2#4)}%
            \Exchange{%
              \DifferentFilecontentsDifferentMessage{##3}{##6}%
              \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
              {##3}#2#4}%
            }%
          }{%
            %\message{1-1-2-2}%
            %\message{Writing:(#3{##3}##4#2#4)}%
            \Exchange{%
              \DifferentFilecontentsDifferentMessage{##3}{##6}%
              \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
              {##3}##4#2#4}%
            }%
          }%
        }%
      }{%
        %\message{1-2}%
        \immediate\read\filecompareread to\templine%
        \expandafter\PassFirstToSecond\expandafter{\templine}{%
          \DifferentFilecontentsCompareLoop{##1}{##2}{##3}{##4}%
        }{##6}%
      }%
    }{%
      %\message{2}%
      \UD@CheckWhetherNull{##1}{%
        %\message{2-1}%
        \UD@CheckWhetherNull{##4}{%
          %\message{2-1-1}%
          %\message{Writing:(#3{##3}#2#4)}%
          \Exchange{%
            \DifferentFilecontentsDifferentMessage{##3}{##6}%
            \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
            {##3}#2#4}%
          }%
        }{%
          %\message{2-1-2}%
          %\message{Writing:(#3{##3}##4#2#4)}%
          \Exchange{%
            \DifferentFilecontentsDifferentMessage{##3}{##6}%
            \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
            {##3}##4#2#4}%
          }%
        }%
      }{%
        %\message{2-2}%
        %\edef\tempa{\romannumeral0\UD@ExtractFirstArgLoop{##5\UD@SelDOm}}%
        %\message{\detokenize\expandafter{\tempa}}%
        %\edef\tempb{\romannumeral0\UD@ExtractFirstArgLoop{##1\UD@SelDOm}}%
        %\message{\detokenize\expandafter{\tempb}}%
        \expandafter\UD@StringifySecond\expandafter\Exchange%
        \romannumeral0\UD@ExtractFirstArgLoop{##5\UD@SelDOm}{%
          \expandafter\UD@StringifySecond\expandafter\Exchange%
          \romannumeral0\UD@ExtractFirstArgLoop{##1\UD@SelDOm}{\ifx}%
        }%
        \expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi%
        {%
          %\message{2-2-1}%
          \expandafter\PassFirstToSecond\expandafter{\@gobble##5}{%
            \expandafter\DifferentFilecontentsCompareLoop\expandafter{\@gobble##1}{##2}{##3}{##4}%
          }{##6}%
        }{%
          %\message{2-2-2}%
          \UD@CheckWhetherNull{##4}{%
            %\message{2-2-2-1}%
            %\message{Writing:(#3{##3}#2#4)}%
            \Exchange{%
              \DifferentFilecontentsDifferentMessage{##3}{##6}%
              \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
              {##3}#2#4}%
            }%
          }{%
            %\message{2-2-2-2}%
            %\message{Writing:(#3{##3}##4#2#4)}%
            \Exchange{%
              \DifferentFilecontentsDifferentMessage{##3}{##6}%
              \begingroup\newlinechar=\endlinechar\scantokens{\endgroup#3##2%
              {##3}##4#2#4}%
            }%
          }%
        }%
      }%
    }%
  }%
}%
\expandafter\PassFirstToSecond\expandafter{\filecontentsend}{%
  \expandafter\PassFirstToSecond\expandafter{\filecontentsbegin}{%
    \expandafter\PassFirstToSecond\expandafter{\delimitersbehind}{%
      \@DifferentFilecontents%
    }{^^M}%
  }%
}%


\makeatother



\begin{document}

%--------------------------------------------------------------------

\begin{DifferentFilecontents}%[overwrite]%
      {myfile.txt}
This is my file.

This is my file.
\end{DifferentFilecontents}

\section{The first variant of myfile.txt}
\IfFileExists{myfile.txt}{\input{myfile.txt}}{Does not exist!}

%--------------------------------------------------------------------

\begin{DifferentFilecontents}%[overwrite]%
      {myfile.txt}
This is my file.

This is my file.
\end{DifferentFilecontents}

\section{Once more the first variant of myfile.txt}
\IfFileExists{myfile.txt}{\input{myfile.txt}}{Does not exist!}

%--------------------------------------------------------------------

\begin{DifferentFilecontents}%[overwrite]%
      {myfile.txt}
This is my file - second variant.

This is my file - second variant.
\end{DifferentFilecontents}

\section{myfile.txt -- the second variant}
\IfFileExists{myfile.txt}{\input{myfile.txt}}{Does not exist!}

\end{document}

The console-output:

$ pdflatex test.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) (preloaded format=pdflatex).
entering extended mode
(./test.tex
LaTeX2e <2018-12-01>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2018/09/03 v1.4i Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/filecontents/filecontents.sty)
(./test.aux)

LaTeX Information: Seems either the file `myfile.txt' does not exist
                   or its content does not equal the content of environment
                   DifferentFilecontents on input lines 250 - 254.
                   The file will be overwritten/created anew.


LaTeX Warning: Overwriting file `./myfile.txt'.


(./myfile.txt)

LaTeX Information: Seems the content of the existing file `myfile.txt'
                   equals the content of environment DifferentFilecontents
                   on input lines 262 - 266.
                   The file will not be overwritten/will not be created anew.

(./myfile.txt)

LaTeX Information: Seems either the file `myfile.txt' does not exist
                   or its content does not equal the content of environment
                   DifferentFilecontents on input lines 274 - 278.
                   The file will be overwritten/created anew.


LaTeX Warning: Overwriting file `./myfile.txt'.


(./myfile.txt) [1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
(./test.aux) )</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmb
x12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb
>
Output written on test.pdf (1 page, 25863 bytes).
Transcript written on test.log.

Image of the resulting .pdf-file:

As you can see on the console-output, LaTeX 2ε-release 2018-12-01, which is older than LaTeX 2ε-release v1.3c, 2019/09/11, was used for compiling the example. The filecontents*-environment of this older release does not process optional arguments. Thus in the example optional arguments are commented out and the package filecontents is loaded for turning the filecontents*-environment into something that can not only be used in the preamble but can be used within the document-environment also and that also does overwrite existing files.