We propose and analyze a generalization of the Kitaev chain for fermions with long-range $p$-wave pairing, which decays with distance as a power-law with exponent $alpha$. Using the integrability of the model, we demonstrate the existence of two types of gapped regimes, where correlation functions decay exponentially at short range and algebraically at long range ($alpha > 1$) or purely algebraically ($alpha < 1$). Most interestingly, along the critical lines, long-range pairing is found to break conformal symmetry for sufficiently small $alpha$. This is accompanied by a violation of the area law for the entanglement entropy in large parts of the phase diagram in the presence of a gap, and can be detected via the dynamics of entanglement following a quench. Some of these features may be relevant for current experiments with cold atomic ions.