Different numbering scheme per chapter

Question

I would like to change the numbering scheme for my theorems per chapter. That's because I have smaller and bigger chapters. Bigger chapters need subsections for proper structuring, and so my theorems are numbered within subsections. For smaller chapters there won't be any subsections so I would like to have my theorems numbered within sections. (That means one less number).

What I'm trying to accomplish is:

Chapter 1.

Section 1.1.

Subsection 1.1.1.

Theorem 1.1.1.1.
Theorem 1.1.1.2.

Chapter 2.

Section 2.1.

Theorem 2.1.1.
Theorem 2.1.2.

How can I do that?

MWE: (randomize section titles and theorem body text as you please)

\documentclass{book}
\usepackage{amsthm}
\newtheorem{theorem}{Theorem}[subsection]
\begin{document}
\chapter{}
\section{}
\subsection{}
\begin{theorem}\end{theorem}
\chapter{}
\section{}
\begin{theorem}\end{theorem}
\begin{theorem}\end{theorem}
\end{document}

Note: egreg's answer includes a version incorporating the hack he gave me here.

Answer 1

If all theorem-like environments share this numbering system, then the following should work. Each \snewtheorem declarations defines two theorem-like environments and another environment; the "subsection" related theorem-like environment is chosen if the counter value \subsection has a positive value. So all theorems should be stated under a section or a subsection, but this seems what you need.

\documentclass{book}
\usepackage{amsthm,xparse}
\NewDocumentCommand{\snewtheorem}{m o m}
 {
  \IfNoValueTF{#2}
   {
    \newtheorem{sec#1}{#3}[section]
    \newtheorem{subsec#1}{#3}[subsection]
   }
   {
    \newtheorem{sec#1}[sec#2]{#3}
    \newtheorem{subsec#1}[subsec#2]{#3}
   }
  \newenvironment{#1}
   {\ifnum\value{subsection}>0
      \csname subsec#1\expandafter\endcsname
    \else
      \csname sec#1\expandafter\endcsname
    \fi}
   {\ifnum\value{subsection}>0 
      \csname endsubsec#1\expandafter\endcsname
    \else
      \csname endsec#1\expandafter\endcsname
    \fi}
}

\snewtheorem{thm}{Theorem}
\snewtheorem{lem}[thm]{Lemma}

\begin{document}
\mainmatter
\chapter{Long}
\section{First}
\subsection{A}
\begin{lem}[Big]\label{insubsec}
This is a big lemma.
\end{lem}

\begin{thm}[Big]
This is a big theorem.
\end{thm}

\subsection{B}
\begin{lem}
This is a lemma.
\end{lem}

\begin{thm}
This is a theorem.
\end{thm}

\chapter{Short}
\section{First}
\begin{lem}[Big]
This is a big lemma.
\end{lem}

\begin{thm}[Big]\label{insec}
This is a big theorem.
\end{thm}

\section{Second}
\begin{lem}
This is a theorem.
\end{lem}

\begin{thm}
This is a theorem.
\end{thm}

\ref{insubsec}

\ref{insec}

\end{document}

---

An extension of the \xnewtheorem I defined time ago for a question you posed about supporting hyperref and \autoref can be like this.

\documentclass{book}

\usepackage{amsthm,xparse,aliascnt,hyperref,bookmark}

\NewDocumentCommand{\xnewtheorem}{m o m}
 {%
  \IfNoValueTF{#2}
   {%
    \newtheorem{sec#1}{#3}[section]%
    \newtheorem{subsec#1}{#3}[subsection]%
   }
   {%
    \newaliascnt{sec#1}{sec#2}%
    \newtheorem{sec#1}[sec#1]{#3}%
    \aliascntresetthe{sec#1}%
    \newaliascnt{subsec#1}{subsec#2}%
    \newtheorem{subsec#1}[subsec#1]{#3}%
    \aliascntresetthe{subsec#1}%
   }
  \expandafter\newcommand\csname sec#1autorefname\endcsname{#3}%
  \expandafter\newcommand\csname subsec#1autorefname\endcsname{#3}%
  \newenvironment{#1}
   {\ifnum\value{subsection}>0
      \csname subsec#1\expandafter\endcsname
    \else
      \csname sec#1\expandafter\endcsname
    \fi}
   {\ifnum\value{subsection}>0 
      \csname endsubsec#1\expandafter\endcsname
    \else
      \csname endsec#1\expandafter\endcsname
    \fi}
 }

\xnewtheorem{thm}{Theorem}
\xnewtheorem{lem}[thm]{Lemma}

\begin{document}
\mainmatter
\chapter{Long}
\section{First}
\subsection{A}
\begin{lem}[Big]\label{insubsec}
This is a big lemma.
\end{lem}

\begin{thm}[Big]
This is a big theorem.
\end{thm}

\subsection{B}
\begin{lem}
This is a lemma.
\end{lem}

\begin{thm}
This is a theorem.
\end{thm}

\chapter{Short}
\section{First}
\begin{lem}[Big]
This is a big lemma.
\end{lem}

\begin{thm}[Big]\label{insec}
This is a big theorem.
\end{thm}

\section{Second}
\begin{lem}
This is a theorem.
\end{lem}

\begin{thm}
This is a theorem.
\end{thm}

\autoref{insubsec}

\autoref{insec}

\end{document}