In strongly correlated organic materials it has been pointed out that charge-ordering could also achieve electronic ferroelectricity at the same critical temperature $T_{co}$. A prototype of such phenomenon are the quasi-one dimensional (TMTTF)$_2X$ Fabre-salts. However, the stabilization of a long-range ferroelectric ground-state below $T_{co}$ requires the break of inversion symmetry, which should be accompanied by a lattice deformation. In this work we investigate the role of the monovalent counter-anion $X$ in such mechanism. For this purpose, we measured the quasi-static dielectric constant along the $c^{*}$-axis direction, where layers formed by donors and anions alternate. Our findings show that the ionic charge contribution is three orders of magnitude lower than the intra-stack electronic response. The $c^{*}$ dielectric constant ($epsilon_{c^*}$) probes directly the charge response of the monovalent anion $X$, since the anion mobility in the structure should help to stabilize the ferroelectric ground-state. Furthermore, our $epsilon_{c^*}$ measurements %conjugated with earlier investigations of the $c^*$ lattice thermal expansion, show that the dielectric response is thermally broaden below $T_{co}$ if the ferroelectric transition occurs in the temperature range where the anion movement begin to freeze in their methyl groups cavity. In the extreme case of the PF$_6$-H$_{12}$ salt, where $T_{co}$ occurs at the freezing point, a relaxor-type ferroelectricity is observed. Also, because of the slow kinetics of the anion sub-lattice, global hysteresis effects and reduction of the charge response upon successive cycling are observed. In this context, we propose that anions control the order-disorder or relaxation character of the ferroelectric transition of the Fabre-salts.