TeX has the concept of an invalid character (a byte sequence with
\catcode 15) so if TeX is being set up with any 8 bit character having this catcode then obviously a format doing so will choke on a file containing this character.
By default (and probably for historical reasons) plain TeX and LaTeX have one character of that nature: "delete" (which is
^^K), so if you have this one in your file you get
! Text line contains an invalid character.
But of course, once you changed the catcode the byte sequence could appear and if you do it as part of the format it could appear always, so from that perspective no byte sequence is always invalid.