indentation problem in self defined environment

Question

I have defined an environment called exercise as follows:

\newcounter{exercise}
\NewDocumentEnvironment{exercise}{o +b}%
  {%
    \IfNoValueTF{#1}
                {\stepcounter{exercise}}
                {\setcounter{exercise}{#1}}%
    \par\noindent\makebox[2em][l]{\textbf{\theexercise.}}%
    \hangindent = 3em \hangafter = 1 #2\par}%
  {\ignorespacesafterend}

And I test it as follows:

\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{enumitem}
\usepackage{lipsum}
\newcounter{exercise}
\NewDocumentEnvironment{exercise}{o +b}%
  {%
    \addvspace{\topsep}%
    \IfNoValueTF{#1}
                {\stepcounter{exercise}}
                {\setcounter{exercise}{#1}}%
    \par\noindent\makebox[2em][l]{\textbf{\theexercise.}}%
    \hangindent = 3em \hangafter = 1 #2\par}%
  {\ignorespacesafterend}

\begin{document}

\begin{exercise}[8]
\lipsum[1][1-2]
\end{exercise}

\begin{exercise}
\lipsum[1][3-4]
\end{exercise}

\begin{exercise}[11]
  \lipsum[1][5-6]
  \begin{enumerate}[label = \textbf{\alph*.}]
    \item \lipsum[2][1-2]
    \item \lipsum[2][3-4]
    \item \lipsum[2][5]
  \end{enumerate}
\end{exercise}

\end{document}

Output:

The vertical red line is added by myself. My question is how to refine the code such that the enumerate part can automatically start from the right side of the line (i.e., to make it also have a corresponding indentation)?

Answer 1

I think this produces what you want:

\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{enumitem}
\usepackage{lipsum}
\newcounter{exercise}
% Courtesy: https://tex.stackexchange.com/a/278219
% Thanks Zarko!
\newcommand*{\entrylabel}[1]{%
  \makebox[\labelwidth][l]{%
    \parbox[t]{\labelwidth}{\raggedright#1}%
  }%
}
\NewDocumentEnvironment{exercise}{ o }{%
  \IfNoValueTF{#1}%
              {\stepcounter{exercise}}%
              {\setcounter{exercise}{#1}}%
  \list{\textbf{\theexercise.}}{%
    \renewcommand{\makelabel}{\entrylabel}%
    \setlength{\labelwidth}{2em}%
    \setlength{\labelsep}{1em}%
    \setlength{\leftmargin}{2em}%
    \setlength{\itemindent}{-1em}%
  }%
  \item
}%
{%
  \endlist
}

\begin{document}
\begin{exercise}[11]
  \lipsum[1][5-6]
  
  \begin{enumerate}[label = \textbf{\alph*.}]
  \item \lipsum[2][1-2]
  \item \lipsum[2][3-4]
  \item \lipsum[2][5]
  \end{enumerate}
\end{exercise}

\begin{exercise}[111]
\lipsum[1][1-2]
\end{exercise}

\begin{exercise}
\lipsum[1][3-4]
\end{exercise}
\end{document}

Note that there are some changes:

1. I don't use +b parameter at all. It can be skipped. This is better as it can involve verbatim text.
2. I shift to the list mechanism of LaTeX.
3. I have reduced your code significantly by removing indentation commands, they are handled well by LaTeX lists.

Answer 2

Use the left key that marks the start of the label and the start of the text (3em and 5.5em, both relative to a lateral margin).

Then with \itemindent = -1em (= 2em - 3em) you will get the same second line indentation of the numbered paragraph.

\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{enumitem}
\usepackage{lipsum}
\newcounter{exercise}
\NewDocumentEnvironment{exercise}{o +b}%
{%% \setlist added <<<<<<<<<<<<<<<<
    \setlist[enumerate]{label = \textbf{\alph*.},
        align=left,%<<<
        itemindent=-1.0em, % 2em-3em <<<<<<<<<<<
        left=3em .. 5.5em}% start of the label .. start of the text <<<<<<<<<<
    \addvspace{\topsep}%
    \IfNoValueTF{#1}
    {\stepcounter{exercise}}
    {\setcounter{exercise}{#1}}%
    \par\noindent\makebox[2em][l]{\textbf{\theexercise.}}%
    \hangindent = 3em  \hangafter = 1 #2\par     
}%
{\ignorespacesafterend}


\usepackage{showframe}% ONLY to show the margins <<<<<<

\begin{document}
    
    \begin{exercise}[8]
        \lipsum[1][1-2]
    \end{exercise}
    
    \begin{exercise}
        \lipsum[1][3-4]
    \end{exercise}
    
    \begin{exercise}[11]
        \lipsum[1][5-6]
        \begin{enumerate}
            \item \lipsum[2][1-2]
            \item \lipsum[2][3-4]
            \item \lipsum[2][5] 
        \end{enumerate}
    \end{exercise}

\end{document}