The stability of the ideal magnetohydrodynamic (MHD) interchange mode at marginal conditions is studied. A sufficiently strong constant magnetic field component transverse to the direction of mode symmetry provides the marginality conditions. A systematic perturbation analysis in the smallness parameter, $|b_2/B_c|^{1/2}$, is carried out, where $B_c$ is the critical transverse magnetic field for the zero-frequency ideal mode, and $b_2$ is the deviation from $B_c$. The calculation is carried out to third order including nonlinear terms. It is shown that the system is nonlinearly unstable in the short wavelength limit, i.e., a large enough perturbation results in instability even if $b_2/B_c>0$ (linearly stable). The normalized amplitude for instability is shown to scale as $|b_2/B_c|^{1/2}$. A nonlinear, compressible, MHD simulation is done to check the analytic result. Good agreement is found, including the critical amplitude scaling.