We obtained near-infrared long-baseline interferometry of IRC+10420 with the AMBER instrument of ESOs Very Large Telescope Interferometer (VLTI) in low and high spectral resolution (HR) mode to probe the photosphere and the innermost circumstellar environment of this rapidly evolving yellow hypergiant. In the HR observations, the visibilities show a noticeable drop across the Brackett gamma (BrG) line on all three baselines, and we found differential phases up to -25 degrees in the redshifted part of the BrG line and a non-zero closure phase close to the line center. The calibrated visibilities were corrected for AMBERs limited field-of-view to appropriately account for the flux contribution of IRC+10420s extended dust shell. We derived FWHM Gaussian sizes of 1.05 +/- 0.07 and 0.98 +/- 0.10 mas for IRC+10420s continuum-emitting region in the H and K bands, respectively, and the BrG-emitting region can be fitted with a geometric ring model with a diameter of 4.18 +0.19/-0.09 mas, which is approximately 4 times the stellar size. The geometric model also provides some evidence that the BrG line-emitting region is elongated towards a position angle of 36 degrees, well aligned with the symmetry axis of the outer reflection nebula. The HR observations were further analyzed by means of radiative transfer modeling using CMFGEN and the 2-D Busche & Hillier codes. Our spherical CMFGEN model poorly reproduces the observed line shape, blueshift, and extension, definitively showing that the IRC+10420 outflow is asymmetric. Our 2-D radiative transfer modeling shows that the blueshifted BrG emission and the shape of the visibility across the emission line can be explained with an asymmetric bipolar outflow with a high density contrast from pole to equator (8-16), where the redshifted light is substantially diminished.