As has been noted in other responses, and in links provided, it is easy enough to shrink margins and choose better fonts. However, the PDF reader software, available on phones and other devices, may have shortcomings. I can read PDF on my old iPod Touch v4 (yes, v4) using the Foxit app. Tiny screen, but works.
One general problem is that, in most cases, TeX does not place the space characters in the PDF. Words are separated by distance between them, not by an intervening space character. Some PDF readers understand that the distance is to be interpreted as a space, but others do not. This makes a big difference if the file is to be audibly read by having software parse the words. It also makes a big difference if the document is allowed to reflow, rather than simply zooming in by scale. Some software allows you to "force" reflow in PDF files that are not themselves set for reflow.
Currently, compiling with pdfLaTeX allows you to specify that spaces will be placed (they are drawn from a special font). But LuaLaTeX does not have this capability. I do not know about XeTeX.
Keep in mind that, as small screen size, the typographic merits of TeX will largely be obliterated. With reflow, they will actually be ignored. The main reason for using TeX for an ebook PDF is if you need equations and diagrams. But if the book is primarily text, you are better off simply doing it directly in XHTML (which converts easily to native ebook formats, rather than PDF). In this case, you will have little control over fonts, and even less control over typography.
Also, if your purpose is to sell licensed copies of your writing, then it will need to be in the ebook format marketed by the major providers. There is no easy way to attach DRM to a PDF (not for amateurs, that is). Existing methods either involve special software or a special server, and are a bit of a nuisance to general readers.