The compounds BaDy$_2$O$_4$ and BaHo$_2$O$_4$ are part of a family of frustrated systems exhibiting interesting properties, including spin liquid-type ground states, magnetic field-induced phases, and the coexistence of short- and long-range magnetic orders, with dominant one-dimensional correlations, which can be described as Ising $J_1-J_2$ zigzag chains along the $c$-axis. We have investigated polycrystalline samples of BaDy$_2$O$_4$ and BaHo$_2$O$_4$ with both neutron diffraction and neutron spectroscopy, coupled to detailed crystalline electric field calculations. The latter points to site-dependent anisotropic magnetism in both materials, which is corroborated by the magnetic structures we determined. The two systems show the coexistence of two different long-range orders --- two double Neel $uparrowuparrowdownarrowdownarrow$ orders in the $ab$-plane with propagation vectors $mathbf{k}_1$ = ($frac{1}{2}$,0,$frac{1}{2}$) and $mathbf{k}_2$ = ($frac{1}{2}$,$frac{1}{2}$,$frac{1}{2}$) for BaDy$_2$O$_4$, and two distinct arrangements of simple Neel $uparrowdownarrowuparrowdownarrow$ orders along the $c$-axis, both with the propagation vector $mathbf{k}_0$ = (0,0,0) for BaHo$_2$O$_4$. The order for both wave vectors in BaDy$_2$O$_4$ occurs at $T_mathrm{N}$ = 0.48 K, while in BaHo$_2$O$_4$, the first order sets in at $T_mathrm{N}sim$ 1.3 K and the second one has a lower ordering temperature of 0.84 K. Both compounds show extensive diffuse scattering which we successfully modeled with a one-dimensional axial next-nearest neighbor Ising (ANNNI) model. In both materials, strong diffusive scattering persists to temperatures well below where the magnetic order is fully saturated. The ANNNI model fits indicate the presence of sites which do not order with moments in the $ab$-plane.