First off, I would use inline-fraction notation, 1/t
, instead of \frac{1}{t}
in order to typeset the matrix more compactly. If you're OK with using psmallmatrix
instead of pmatrix
, you can then get by with \bigl(
, \bigm|
(or \bigm\vert
), and \bigr)
. If you prefer using pmatrix
, use \biggl(
, \biggm|
, and \biggr)
instead.

Observe that \biggm|
automatically adds the typographically required amount of whitespace around the vertical bar. This is in contrast to \bigg|
, wwhere no whitespace is inserted.
\documentclass{article}
\usepackage{mathtools} % for 'psmallmatrix' env.
\usepackage{amssymb} % for '\mathbb' macro
\begin{document}
\begin{align*}
G&=\bigl\{
\begin{psmallmatrix} t & 0\\ 0 & 1/t \end{psmallmatrix}
\bigm|
t\in\mathbb{R}^{\times}
\bigr\} \\
&=\biggl\{
\begin{pmatrix} t & 0\\ 0 & 1/t \end{pmatrix}
\biggm|
t\in\mathbb{R}^{\times}
\biggr\}
\end{align*}
\end{document}