Neutron stars in low mass X-ray binaries are hypothesised to emit continuous gravitational waves that may be detectable by ground-based observatories. The torque balance model predicts that a higher accretion rate produces larger-amplitude gravitational waves, hence low mass X-ray binaries with high X-ray flux are promising targets for gravitational wave searches. The detection of X-ray pulsations would identify the spin frequency of these neutron stars, and thereby improve the sensitivity of continuous gravitational-wave searches by reducing the volume of the search parameter space. We perform a semi-coherent search for pulsations in the two low mass X-ray binaries Scorpius X-1 and Cygnus X-2 using X-ray data from the textit{ Rossi X-ray Timing Explorer} Proportional Counter Array. We find no clear evidence for pulsations, and obtain upper limits (at $90%$ confidence) on the fractional pulse amplitude, with the most stringent being $0.034%$ for Scorpius X-1 and $0.23%$ for Cygnus X-2. These upper limits improve upon those of Vaughan et al. (1994) by factors of $sim 8.2$ and $sim 1.6$ respectively.