We propose and analyse a pumped Bose-Hubbard dimer as a source of continuous-variable Einstein-Podolsky-Rosen (EPR) steering with non-Gaussian statistics. We use the truncated Wigner representation to calculate third and fourth order cumulants, finding clear signals of non-Gaussianity. We also calculate the products of inferred quadrature variances which indicate that states demonstrating the EPR paradox are present. Our proposed experimental configuration is extrapolated from current experimental techniques and adds another possibility to the current toolbox of quantum atom optics.