Compounds with two-dimensional (2D) layers of magnetic ions weakly connected by van der Waals bonding offer routes to enhance quantum behavior, stimulating both fundamental and applied interest. CrPS4 is one such magnetic van der Waals material, however, it has undergone only limited investigation. Here we present a comprehensive series of neutron scattering measurements to determine the magnetic structure and exchange interactions. The observed magnetic excitations allow a high degree of constraint on the model parameters not normally associated with measurements on a powder sample. The results demonstrate the 2D nature of the magnetic interactions, while also revealing the importance of interactions along 1D chains within the layers. The subtle role of competing interactions is observed, which manifest in a non-trivial magnetic transition and a tunable magnetic structure in a small applied magnetic field through a spin-flop transition. Our results on the bulk compound provide insights that can be applied to an understanding of the behavior of reduced layer CrPS4.