How can I combine two newtheorem environments so that one is continuing the numbering of the other plus some additional numbering?

Question

I want to achieve having two different environments for my lemmas, say lem1 and lem2, such that Lemmas written by \begin{lem1} \end{lem1} are numbered within the section, and if I add lemmas by \begin{lem2} \end{\lem2} they pick up the numbering of lem1 but with an additional numbering within consecutive lem2-lemmas until the next lem1-lemma comes.

Hope this does not sound too confusing, but I think an example clarifies pretty well what I mean:

1. Section

2. Section

   2.1. Subsection 

      Lemma 2.1             <- lem1

   2.2. Subsection

      Lemma 2.2             <- lem1

      Lemma 2.3             <- lem1

      Lemma 2.4 (I)         <- lem2

      Lemma 2.4 (II)        <- lem2

      Lemma 2.5             <- lem1

      Lemma 2.6 (I)         <- lem2

      Lemma 2.6 (II)        <- lem2

I have achieved something similar but not quite what I want by:

\newtheorem{lemma1}{Lemma}
\newtheorem{lem1}[lemma1]{Lemma}
\numberwithin{lemma1}{section}

\newtheorem{lemma2}{Lemma}
\newtheorem{lem2}[lemma2]{Lemma}
\numberwithin{lemma2}{section}
\renewcommand{\thelemma2}{\arabic{section}.\arabic{lemma1} (\Roman{lemma2})}

This leads to

1. Section

2. Section

   2.1. Subsection 

      Lemma 2.1             <- lem1

   2.2. Subsection

      Lemma 2.2             <- lem1

      Lemma 2.3             <- lem1

      Lemma 2.3 (I)         <- lem2

      Lemma 2.3 (II)        <- lem2

      Lemma 2.4             <- lem1

      Lemma 2.4 (III)       <- lem2

      Lemma 2.4 (IV)        <- lem2

Can someone please help me? :-)

Thank you in advance

Edit for clarification: I need that Lemma 2.3 is followed by 2.4 (I)

Edit after having read the proposed solution: The solution also works when having defined

\newtheorem{firsttheorem}{Proposition}
\numberwithin{firsttheorem}{section}

with lemma (in the solution) replaced by firsttheorem. Then one can write

\begin{sublemmas}

    \begin{whatever is defined as firsttheorem}

    \end{whatever is defined as firsttheorem}

\end{sublemmas}

and everything works perfectly!

Answer 1

You need to tell LaTeX when to start subnumbering and when end it.

The idea here is the same as subequations.

\documentclass{article}
\usepackage{amsthm}

\newtheorem{lemma}{Lemma}[section]

\newcounter{parentlemma}
\newenvironment{sublemmas}
 {%
  \refstepcounter{lemma}%
  \setcounter{parentlemma}{\value{lemma}}%
  \edef\theparentlemma{\thelemma}%
  \setcounter{lemma}{0}%
  \renewcommand{\thelemma}{\theparentlemma\space(\Roman{lemma})}%
  \ignorespaces
 }
 {\setcounter{lemma}{\value{parentlemma}}\ignorespacesafterend}

\begin{document}

\section{Test A}

\begin{lemma}
This is a lemma
\end{lemma}

\section{Test B}

\begin{lemma}
This is a lemma
\end{lemma}

\begin{lemma}
This is a lemma
\end{lemma}

\begin{sublemmas}
\begin{lemma}
This is a lemma
\end{lemma}

\begin{lemma}
This is a lemma
\end{lemma}
\end{sublemmas}

\begin{lemma}
This is a lemma
\end{lemma}

\begin{sublemmas}
\begin{lemma}
This is a lemma
\end{lemma}

\begin{lemma}
This is a lemma
\end{lemma}
\end{sublemmas}

\end{document}

You can use \label after \begin{sublemmas} to get a reference to the common number. And also arbitrary text between the sublemmas, of course.