# latex notation for matrix inside matrix

could anyone help me how to write in latex a matrix like the one below? I tried for the first one but didn't get exact form.

$$\begin{bmatrix} A&0&\dots\\ B&A&\dots\\ 0&B&\dots\\ \dots&\dots&\dots\\ \dots&\dots&A\\ \dots&\dots&B\\ \end{bmatrix}$$


Here is a solution with pstricks. Compile with xelatex, or with pdflatex if you use the --enable-write18 switch (MiKTeX) ot -shell-escape (TeX Live, MacTeX).

\documentclass{article}
\usepackage{mathtools}
\usepackage{mathptmx}
$% \setlength\fboxrule{0pt}\setlength\fboxsep{5pt} \setlength\arraycolsep{6pt} \begin{postscript} \begin{bmatrix} \pnode(0.2em,2ex){a} \rnode{11}{A}& \rnode{12}{0} & & & \rnode{13}{0}\pnode(-0.2em,2ex){b}\\ B&\rnode{22}{A} \\ \rnode{31}{0}&\rnode{32}{B} \\ \\ & & & & \rnode{43}{A}\\ \rnode{51}{0} & \rnode{52}{0} & & & \rnode{53}{B}\\ \end{bmatrix} \psset{linestyle=dotted, dotsep=7pt, nodesep=9pt} % horizontal dots \ncline[offset=-0.7ex]{12}{13}\ncline[offset=-0.9ex]{52}{53} % vertical dots \ncline[nodesepB=4pt]{31}{51}\ncline[nodesepB=4pt]{32}{52}\ncline[dotsep=8pt, nodesepA =14pt]{13}{43} %diagonal dots \psset{nodesep=3pt, dotsep=8pt} \ncline{22}{43}\ncline{32}{53} \psset{braceWidth=0.8pt, braceWidthInner=4pt, braceWidthOuter=4pt, } \psbrace*[rot=-90](b)(a){\clap{\fbox{\scriptstyle \varepsilon{+}1 \enspace}}} \end{postscript}$%