We investigate the effect of a strong magnetic field on the structure of neutron stars in a model with perturbative $f(R)$ gravity. The effect of an interior strong magnetic field of about $10^{17 sim 18}$ G on the equation of state is derived in the context of a quantum hadrodynamics (QHD) model. We solve the modified spherically symmetric hydrostatic equilibrium equations derived for a gravity model with $f(R)=R+alpha R^2$. Effects of both the finite magnetic field and the modified gravity are detailed for various values of the magnetic field and the perturbation parameter $alpha$ along with a discussion of their physical implications. We show that there exists a parameter space of the modified gravity and the magnetic field strength, in which even a soft equation of state can accommodate a large ($> 2$ M$_odot$) maximum neutron star mass through the modified mass-radius relation.