We study an inflationary scenario with a two-form field to which an inflaton couples non-trivially. First, we show that anisotropic inflation can be realized as an attractor solution and that the two-form hair remains during inflation. A statistical anisotropy can be developed because of a cumulative anisotropic interaction induced by the background two-form field. The power spectrum of curvature perturbations has a prolate-type anisotropy, in contrast to the vector models having an oblate-type anisotropy. We also evaluate the bispectrum and trispectrum of curvature perturbations by employing the in-in formalism based on the interacting Hamiltonians. We find that the non-linear estimators $f_{NL}$ and $tau_{NL}$ are correlated with the amplitude $g_*$ of the statistical anisotropy in the power spectrum. Unlike the vector models, both $f_{NL}$ and $tau_{NL}$ vanish in the squeezed limit. However, the estimator $f_{NL}$ can reach the order of 10 in the equilateral and enfolded limits. These results are consistent with the latest bounds on $f_{NL}$ constrained by Planck.