Magnetic ordering phenomena have a profound influence on the macroscopic properties of correlated-electron materials, but their realistic prediction remains a formidable challenge. An archetypical example is the ternary nickel oxide system RNiO3 (R = rare earth), where the period-four magnetic order with proposals of collinear and non-collinear structures and the amplitude of magnetic moments on different Ni sublattices have been subjects of debate for decades. Here we introduce an elementary model system - NdNiO3 slabs embedded in a non-magnetic NdGaO3 matrix - and use polarized resonant x-ray scattering (RXS) to show that both collinear and non-collinear magnetic structures can be realized, depending on the slab thickness. The crossover between both spin structures is correctly predicted by density functional theory and can be qualitatively understood in a low-energy spin model. We further demonstrate that the amplitude ratio of magnetic moments in neighboring NiO6 octahedra can be accurately determined by RXS in combination with a correlated double cluster model. Targeted synthesis of model systems with controlled thickness and synergistic application of polarized RXS and ab-initio theory thus provide new perspectives for research on complex magnetism, in analogy to two-dimensional materials created by exfoliation.