We measure the expansion of an ultracold plasma across the field lines of a uniform magnetic field. We image the ion distribution by extracting the ions with a high voltage pulse onto a position-sensitive detector. Early in the lifetime of the plasma ($< 20$ $mu$s), the size of the image is dominated by the time-of-flight Coulomb explosion of the dense ion cloud. For later times, we measure the 2-D Gaussian width of the ion image, obtaining the transverse expansion velocity as a function of magnetic field (up to 70 G). We observe that the expansion velocity scales as B$^{-1/2}$, explained by a nonlinear ambipolar diffusion model with anisotropic diffusion in two different directions.