An accurate assessment of the Suns angular momentum (AM) loss rate is an independent constraint for models that describe the rotation evolution of Sun-like stars. In-situ measurements of the solar wind taken by Parker Solar Probe (PSP), at radial distances of $sim 28-55R_{odot}$, are used to constrain the solar wind AM-loss rate. For the first time with PSP, this includes a measurement of the alpha particle contribution. The mechanical AM flux in the solar wind protons (core and beam), and alpha particles, is determined as well as the transport of AM through stresses in the interplanetary magnetic field. The solar wind AM flux is averaged over three hour increments, so that our findings more accurately represent the bulk flow. During the third and fourth perihelion passes of PSP, the alpha particles contain around a fifth of the mechanical AM flux in the solar wind (the rest is carried by the protons). The proton beam is found to contain $sim 10-50%$ of the proton AM flux. The sign of the alpha particle AM flux is observed to correlate with the proton core. The slow wind has a positive AM flux (removing AM from the Sun as expected), and the fast wind has a negative AM flux. As with previous works, the differential velocity between the alpha particles and the proton core tends to be aligned with the interplanetary magnetic field. In future, by utilising the trends in the alpha-proton differential velocity, it may be possible to estimate the alpha particle contribution when only measurements of the proton core are available. Based on the observations from this work, the alpha particles contribute an additional $10-20%$ to estimates of the solar wind AM-loss rate which consider only the proton and magnetic field contributions. Additionally, the AM flux of the proton beam can be just as significant as the alpha particles, and so should not be neglected in future studies.