# Building a list of non-duplicated labels from text on the fly

Summary of issue: The former question showed some TeX code that used a list of strings which were labels derived from references in the text. However, that list could contain duplicates, and that code did not deduplicate that list. The latter question was focused on deduplicating that list after collecting it from the text.

However, David Carlisle indicated in chat that it might be possible to create a list without duplicates in the first instance. I had assumed it would be faster and easier to create a list with duplicates, and then remove the duplicates, but perhaps not. In any case, I'd be interested in a solution that uses this approach.

The relevant exchange started at

http://chat.stackexchange.com/transcript/message/20082058#20082058

where David wrote

I can't see why you'd need \clist_remove_duplicates:N if duplicates are a problem then just don't put them in the list.

I responded with

@DavidCarlisle well, you'd have to check the list at every addition. One could do that, certainly, but it's probably more efficient to remove dupes at the end.

To which David responded:

@FaheemMitha I doubt it

and followed up with

@FaheemMitha You are adding ref so if \csname r@#1\endcsname is already defined you have seen this ref before and don't need to add it again. There is need to build a list with duplicates

I replied:

You are presumably saying that code in the original question can be modified so dupes are not added.

to which he responded

@FaheemMitha yes although not testing r@#1 (as that just tells you that there is a label) but you could make each ref leave a csname so you know you have seen it (this trades space for time)

You can just define a flag macro (to anything) and then not add a new entry if the macro is already defined.

% VERSION 1
\iftrue
\makeatletter
\let\oldref\ref
\def\ref#1{%
\expandafter\ifx\csname R@#1\endcsname\relax
\global\expandafter\let\csname R@#1\endcsname\@empty
\immediate\write\@auxout{%
\string\gappto\string\ReferencedIDs{#1,}%
}%
\fi
\oldref{#1}%
}
\def\ReferencedIDs{}
\makeatother

\else
% VERSION 2

\makeatletter
\AtBeginDocument{\providecommand\ReferencedIDs{}}
\AtEndDocument{\immediate\write\@auxout{\gdef\string\ReferencedIDs{\ReferencedIDs}}}
\let\oldref\ref
\def\ref#1{%
\expandafter\ifx\csname R@#1\endcsname\relax
\global\expandafter\let\csname R@#1\endcsname\@empty