I wouldn’t use the level macros for the enumeration, that gets messy. ( \part introduces a whole page on its own!)
I rather redefine the enumeration style.
Roman enumeration at the level of chapter
There's no need to patch \thesection (to get “I.1”, “I.2”, …) because \thechapter was used before, anyway (“3.1”, “3.2”, …).
Provided macros
\makeChapterRoman
This macro saves the current value of the chapter counter in the counter chapterBackup, sets the style of \thechapter to Roman (I, II, III, …) and let the enumeration start with 1.
The old definition of \thechapter is saved with \let to \oldThechapter (it’s easier to undo).
\undoChapterRoman
This macro undoes all changes, i.e. using the old chapter value (from chapterBackup) and re-\letting \thechapter to its initial definition.
Code
\documentclass{report}
\newcounter{chapterBackup}
\newcommand*{\makeChapterRoman}{%
\let\oldThechapter\thechapter%
\setcounter{chapterBackup}{\value{chapter}}%
\setcounter{chapter}{0}%
\renewcommand*{\thechapter}{\Roman{chapter}}%
}
\newcommand*{\undoChapterRoman}{%
\setcounter{chapter}{\value{chapterBackup}}%
\let\thechapter\oldThechapter%
}
\begin{document}
\tableofcontents
\chapter{Chapter 1}\chapter{Chapter 2}
\chapter{Chapter 3}
\section{Foo}
\section{Far}
\makeChapterRoman
\chapter{Baz}
\section{Bar}
\chapter{Caz}
\section{Coo}
\undoChapterRoman
\chapter{Back to normal}
\section{Normal section}
\end{document}
Output

Roman enumeration at the level of section
At the default style a subsubsection doesn’t have a enumeration. Even with (the secnumdepth counter being greater than 2) there’s no need to redefine \thesubsubsection because it uses \thesubsection and not the raw \arabic… commands.
Provided macros
\makeSectionRoman
\undoSectionRoman
Code
\documentclass{report}
\newcounter{sectionBackup}
\newcommand*{\makeSectionRoman}{%
\let\oldThesection\thesection%
\let\oldThesubsection\thesubsection%
\setcounter{sectionBackup}{\value{section}}%
\setcounter{section}{0}%
\renewcommand*{\thesection}{\Roman{section}}%
\renewcommand*{\thesubsection}{\thesection.\arabic{subsection}}%
}
\newcommand*{\undoSectionRoman}{%
\setcounter{section}{\value{sectionBackup}}%
\let\thesection\oldThesection%
\let\thesubsection\oldThesubsection%
}
\begin{document}
\tableofcontents
\chapter{Chapter 1}\chapter{Chapter 2}
\chapter{Chapter 3}
\section{Foo}\section{Far}
\makeSectionRoman
\section{Baz}
\subsection{Bar}
\section{Caz}
\subsection{Coo}
\undoSectionRoman
\section{Back to normal}
\subsection{Normal section}
\subsection{Another normal section}
\end{document}
Output

\documentclassare you using?reportorbook, or perhaps something different altogether? – Werner Dec 5 '12 at 3:20