High-resolution thermal expansion measurements have been performed for exploring the mysterious structureless transition in (TMTTF)$_{2}$X (X = PF$_{6}$ and AsF$_{6}$), where charge ordering at $T_{CO}$ coincides with the onset of ferroelectric order. Particularly distinct lattice effects are found at $T_{CO}$ in the uniaxial expansivity along the interstack $textbf{textit{c*}}$-direction. We propose a scheme involving a charge modulation along the TMTTF stacks and its coupling to displacements of the counteranions X$^{-}$. These anion shifts, which lift the inversion symmetry enabling ferroelectric order to develop, determine the 3D charge pattern without ambiguity. Evidence is found for another anomaly for both materials at $T_{int}$ $simeq$ 0.6 $cdot$ $T_{CO}$ indicative of a phase transition related to the charge ordering.