We study the response of the antiferromagnetism of CeAuSb$_2$ to orthorhombic lattice distortion applied through in-plane uniaxial pressure. The response to pressure applied along a $langle 110 rangle$ lattice direction shows a first-order transition at zero pressure, which shows that the magnetic order lifts the $(110)/(1bar{1}0)$ symmetry of the unstressed lattice. Sufficient $langle 100 rangle$ pressure appears to rotate the principal axes of the order from $langle 110 rangle$ to $langle 100 rangle$. At low $langle 100 rangle$ pressure, the transition at $T_N$ is weakly first-order, however it becomes continuous above a threshold $langle 100 rangle$ pressure. We discuss the possibility that this behavior is driven by order parameter fluctuations, with the restoration of a continuous transition a result of reducing the point-group symmetry of the lattice.