I've found practice with LaTeX makes you faster. Now, I've got a writing disability, so writing out a good copy that other people can read takes me a long time, but I find with practice LaTeX isn't that slow. Also, the first assignment in it, getting the preamble set up and whatnot is the slowest part. After that it should be faster; I did a lot of my university homework in LaTeX.
That said, there are almost never marks for it looking good, just being readable, so if it is becoming too large a time sink, abandon ship.
Some advice: Don't worry about making your code pretty. I put one sentence, equation, etc per line to make debugging easier. Sometimes I'll even break up formulas and such to show bracket matching, like you would in a C program or such.
Don't worry about your output being pretty: \left( and \right) may be ugly, but it makes matching them easy when you get errors about them not matching. Ditto for overful hboxes, etc etc.
Write out a 'good enough' copy. This is a copy that removes all the stuff that is scribbled out, where you went down dead paths, etc. But isn't in nice writing or whatnot. Legible, enough, but not pretty as a formal good copy. That way if a deadline is looming, you can say, good enough and just hand that in.
Don't be afraid to be almost right. Don't want to take the time to figure out how to do a specific symbol? Use an easier one, and then define it in text. You see old textbooks do that all the time if they couldn't type R in the fancy real number font. Or if you need to make a figure and it would take a while in LaTeX? Open up Paint or The Gimp or whatever and then insert it as an image.
Compile early, and often, and do so from the command line. That way you'll see where you made a mistake right away, and not have to go hunting for where you made that error. I often keep writing in my text editor while it compiles in the command line window.
Edit: Also, is it really fancy latex, or just basic math formulas? What I've done before is use Word to do the document, then inserted the math from http://www.codecogs.com/latex/eqneditor.php as an image.