We study the intrinsic transport anisotropy and fermiology of the quasi one-dimensional superconductor Ta$_4$Pd$_3$Te$_{16}$. Below $T^*=20$ K we detect a thermodynamic phase transition that predominantly affects the conductivity perpendicular to the quasi one-dimensional chains, consistent with a thermodynamic transition related to the presence of charge order that precedes superconductivity. Remarkably the Fermi surface pockets detected by de Haas-van Alphen (dHvA) oscillations are unaffected by this transition, suggesting that the ordered state does not break any translational symmetries but rather alters the scattering of the quasiparticles themselves.