It actually does not matter very much which document class you use. Most document classes can be adjust to have a different paper size either out of the box or with the help of a package. But even the decision of the document class is at this point not crucial, because you can change it later with little or no effort. Generally, I would use either
scrbook. Both offer many possibilities to adjust the layout to your needs.
You can simply start to write the text, since the content does not/should not depend much on the book design. If you have long equations or tables you may have to rearrange them if your customer decides to use a different page size. What I would not do is to start creating figures. Figures depend strongly on the page size and the overall layout. Which font you use, which (text) colors you use etc.
For your last point: you can take a look into the documentation of
scrbook both have some considerations about the margin sizes, font size and the text width. Also a good resource of information is the memdesign document.