In this paper we study cosmological perturbations in teleparallel gravity. We discuss problems which appear in standard approach to $f(T)$ gravity, and find that these problems may be solved within covariant formulation of teleparallel gravity, which take into account spin connection. We calculate spin connection which symmetrize equation for perturbation and split diagonal and non-diagonal part of equation of motion. We demonstrate that there is minimal solution for spin connection, which lead to zero slip, however, in this case one additional equation appears, so the system may become over-determined. After that, we show that a more general solution exists, which is incompatible with zero slip, but allows to write down the equations of motion for cosmological perturbation in a self-consistent way without additional equations to be satisfied.