We present out-of-plane dielectric and magnetodielectric measurements of single crystallines $alpha$-RuCl$_3$ with various degrees of stack faults. A frequency dependent, but field independent, dielectric anomaly appears at $T_{A}:(f=100:mathrm{kHz})sim$ 4 K once both magnetic transitions at $T_{N1}sim$ 7 K and $T_{N2}sim$ 14 K set in. The observed dielectric anomaly is attributed to the emergency of possible local electric polarizations whose inversion symmetry is broken by inhomogeneously distributed stacking faults. A field-induced intermediate phase is only observed when a magnetic field is applied perpendicular to the Ru-Ru bonds for samples with minimal stacking faults. Less pronounced in-plane anisotropy is found in samples with sizable contribution from stacking imperfections. Our findings suggest that dielectric measurement is a sensitive probe in detecting the structural and magnetic properties, which may be a promising tool especially in studying $alpha$-RuCl$_3$ thin film devices. Moreover, the stacking details of RuCl$_3$ layers strongly affect the ground state both in the magnetic and electric channels. Such a fragile ground state against stacking faults needs to be overcome for realistic applications utilizing the magnetic and/or electric properties of Kitaev based physics in $alpha$-RuCl$_3$.