I use asymptote because of its amazing 3D capabilities. However, when I tried to plot a surface over a nonrectangular domain, I could not do it without getting jagged edges. As far as I know, other software such as Mathematica, Maple, or JavaView allow this kind of plots.
So the question is: is it possible to draw a generic surface in asymptote over a nonrectangular domain in such a way that its edges are smooth?
contour3
module may be helpful. Otherwise, I'm reasonably sure the solution you describe below (mapping the domain onto a rectangle) is the only other currently feasible solution. I suppose you could set the mesh to be extremely fine so that the ragged edges are less obvious, but that would take forever to compile without really solving the problem.