One possibility using the xparse
package to define an environment with the two desired optional arguments:
\documentclass{article}
\usepackage{amsthm}
\usepackage{xparse}
\newcommand\thmcite{}
\newtheoremstyle{mystyle}% name
{\topsep}%Space above
{\topsep}%Space below
{\itshape}%Body font
{0pt}%Indent amount
{\bfseries}% Theorem head font
{:}%Punctuation after theorem head
{\newline}%Space after theorem head 2
{\thmname{#1}~\thmnumber{#2}\thmnote{~(#3)}\thmcite}%Theorem head spec (can be left empty, meaning ‘normal’)
\theoremstyle{mystyle}
\newtheorem{theo}{Theorem}
\DeclareDocumentEnvironment{theorem}{O{}o}
{\IfNoValueTF{#2}{}{\renewcommand\thmcite{~#2}}\begin{theo}[#1]}
{\end{theo}}
\begin{document}
\begin{theorem}[Ordinary differential][\cite{1}]
Test
\end{theorem}
\begin{theorem}[Ordinary differential]
Test
\end{theorem}
\begin{theorem}[][\cite{1}]
Test
\end{theorem}
\begin{theorem}
Test
\end{theorem}
\begin{thebibliography}{9}
\bibitem{1} Author A. Title A. 2013
\end{thebibliography}
\end{document}
If the separation between the head and the body should be increased, one can use an appropriate \vspace{<length>}
in the sixth argument:
\documentclass{article}
\usepackage{amsthm}
\usepackage{xparse}
\newcommand\thmcite{}
\newtheoremstyle{mystyle}% name
{\topsep}%Space above
{\topsep}%Space below
{\itshape}%Body font
{0pt}%Indent amount
{\bfseries}% Theorem head font
{:\vspace{.5\baselineskip}}%Punctuation after theorem head
{\newline}%Space after theorem head 2
{\thmname{#1}~\thmnumber{#2}\thmnote{~(#3)}\thmcite}%Theorem head spec (can be left empty, meaning ‘normal’)
\theoremstyle{mystyle}
\newtheorem{theo}{Theorem}
\DeclareDocumentEnvironment{theorem}{O{}o}
{\IfNoValueTF{#2}{}{\renewcommand\thmcite{~#2}}\begin{theo}[#1]}
{\end{theo}}
\begin{document}
\begin{theorem}[Ordinary differential]%[\cite{1}]
Test
\end{theorem}
\begin{theorem}[Ordinary differential]
Test
\end{theorem}
\begin{theorem}[][\cite{1}]
Test
\end{theorem}
\begin{theorem}
Test
\end{theorem}
\begin{thebibliography}{9}
\bibitem{1} Author A. Title A. 2013
\end{thebibliography}
\end{document}
New requirement made in a comment: to define some other structures in a similar fashion:
\documentclass{article}
\usepackage{amsthm}
\usepackage{xparse}
\newcommand\thmcite{}
\newtheoremstyle{mystyle}% name
{\topsep}%Space above
{\topsep}%Space below
{\itshape}%Body font
{0pt}%Indent amount
{\bfseries}% Theorem head font
{:\vspace{.5\baselineskip}}%Punctuation after theorem head
{\newline}%Space after theorem head 2
{\thmname{#1}~\thmnumber{#2}\thmnote{~(#3)}\thmcite}%Theorem head spec (can be left empty, meaning ‘normal’)
\theoremstyle{mystyle}
\newtheorem{theo}{Theorem}
\newtheorem{lemm}[theo]{Lemma}
\newtheorem{defi}[theo]{Definition}
\DeclareDocumentEnvironment{theorem}{O{}o}
{\IfNoValueTF{#2}{}{\renewcommand\thmcite{~#2}}\begin{theo}[#1]}
{\end{theo}}
\DeclareDocumentEnvironment{lemma}{O{}o}
{\IfNoValueTF{#2}{}{\renewcommand\thmcite{~#2}}\begin{lemm}[#1]}
{\end{lemm}}
\DeclareDocumentEnvironment{definition}{O{}o}
{\IfNoValueTF{#2}{}{\renewcommand\thmcite{~#2}}\begin{defi}[#1]}
{\end{defi}}
\begin{document}
\begin{theorem}[Ordinary differential]
Test theorem.
\end{theorem}
\begin{definition}
Test definition.
\end{definition}
\begin{lemma}[][\cite{1}]
Test lemma.
\end{lemma}
\begin{thebibliography}{9}
\bibitem{1} Author A. Title A. 2013
\end{thebibliography}
\end{document}
The idea is the same. Using \newtheorem
one creates the basic theorem-like structure, and then, with the help of xparse
the corresponding environment with the two required optional arguments is defined.
All three structures share a counter; this can be easily changed (as well as making the counter(s) be subordinated to an already existing counter) using the optional arguments of \newtheorem
(see the amsthm
package documentation for further details).