Crystallographic and magnetic properties of Fe5PB2, Fe4CoPB2, Fe4MnPB2, Fe5SiB2, Fe4CoSiB2, and Fe4MnSiB2 are reported. All adopt the tetragonal Cr5B3 structure-type and are ferromagnetic at room temperature with easy axis of magnetization along the c-axis. The spin reorientation in Fe5SiB2 is observed as an anomaly in the magnetization near 170 K, and is suppressed by substitution of Co or Mn for Fe. The silicides are found to generally have larger magnetic moments than the phosphides, but the data suggests smaller magnetic anisotropy in the silicides. Cobalt substitution reduces the Curie temperatures by more than 100 K and ordered magnetic moments by 16-20%, while manganese substitution has a much smaller effect. This suggests Mn moments align ferromagnetically with the Fe and that Co does not have an ordered moment in these structures. Anisotropic thermal expansion is observed in Fe5PB2 and Fe5SiB2, with negative thermal expansion seen along the c-axis of Fe5SiB2. First principles calculations of the magnetic properties of Fe5SiB2 and Fe4MnSiB2 are reported. The results, including the magnetic moment and anisotropy, and are in good agreement with experiment.