The youngest Galactic supernova remnant (SNR) G1.9+0.3, produced by a (probable) SN Ia that exploded $sim 1900$ CE, is strongly asymmetric at radio wavelengths, much brighter in the north, but bilaterally symmetric in X-rays. We present the results of X-ray expansion measurements that illuminate the origin of the radio asymmetry. We confirm the mean expansion rate (2011 to 2015) of 0.58% per year, but large spatial variations are present. Using the nonparametric Demons method, we measure the velocity field throughout the entire SNR, finding that motions vary by a factor of 5, from 0.09 to 0.44 per year. The slowest shocks are at the outer boundary of the bright northern radio rim, with velocities $v_s$ as low as 3,600 km/s (for an assumed distance of 8.5 kpc), much less than $v_s = 12,000 - 13,000$ km/s along the X-ray-bright major axis. Such strong deceleration of the northern blast wave most likely arises from the collision of SN ejecta with a much denser than average ambient medium there. This asymmetric ambient medium naturally explains the radio asymmetry. In several locations, significant morphological changes and strongly nonradial motions are apparent. The spatially-integrated X-ray flux continues to increase with time. Based on Chandra observations spanning 8.3 years, we measure its increase at 1.3% +/- 0.8% per year. The SN ejecta are likely colliding with the asymmetric circumstellar medium ejected by the SN progenitor prior to its explosion.