We give an algorithm for modular composition of degree n univariate polynomials over a finite field F_q requiring n^{1 + o(1)}log^{1 + o(1)}q bit operations; this had earlier been achieved in characteristic n^{o(1)} by Umans (2008). As an application, we obtain a randomized algorithm for factoring degree n polynomials over F_q requiring (n^{1.5 + o(1)} + n^{1 + o(1)}log q)log^{1 + o(1)}q bit operations, improving upon the methods of von zur Gathen & Shoup (1992) and Kaltofen & Shoup (1998). Our results also imply algorithms for irreducibility testing and computing minimal polynomials whose running times are best-possible, up to lower order terms.As in Umans (2008), we reduce modular composition to certain instances of multipoint evaluation of multivariate polynomials. We then give an algorithm that solves this problem optimally (up to lower order terms), in arbitrary characteristic. The main idea is to lift to characteristic 0, apply a small number of rounds of multimodular reduction, and finish with a small number of multidimensional FFTs. The final evaluations are then reconstructed using the Chinese Remainder Theorem. As a bonus, we obtain a very efficient data structure supporting polynomial evaluation queries, which is of independent interest.Our algorithm uses techniques which are commonly employed in practice, so it may be competitive for real problem sizes. This contrasts with previous asymptotically fast methods relying on fast matrix multiplication.