Gravitational wave searches for continuous-wave signals from neutron stars are especially challenging when the stars spin frequency is unknown a priori from electromagnetic observations and wanders stochastically under the action of internal (e.g. superfluid or magnetospheric) or external (e.g. accretion) torques. It is shown that frequency tracking by hidden Markov model (HMM) methods can be combined with existing maximum likelihood coherent matched filters like the F-statistic to surmount some of the challenges raised by spin wandering. Specifically it is found that, for an isolated, biaxial rotor whose spin frequency walks randomly, HMM tracking of the F-statistic output from coherent segments with duration T_drift = 10d over a total observation time of T_obs = 1yr can detect signals with wave strains h0 > 2e-26 at a noise level characteristic of the Advanced Laser Interferometer Gravitational Wave Observatory (Advanced LIGO). For a biaxial rotor with randomly walking spin in a binary orbit, whose orbital period and semi-major axis are known approximately from electromagnetic observations, HMM tracking of the Bessel-weighted F-statistic output can detect signals with h0 > 8e-26. An efficient, recursive, HMM solver based on the Viterbi algorithm is demonstrated, which requires ~10^3 CPU-hours for a typical, broadband (0.5-kHz) search for the low-mass X-ray binary Scorpius X-1, including generation of the relevant F-statistic input. In a realistic observational scenario, Viterbi tracking successfully detects 41 out of 50 synthetic signals without spin wandering in Stage I of the Scorpius X-1 Mock Data Challenge convened by the LIGO Scientific Collaboration down to a wave strain of h0 = 1.1e-25, recovering the frequency with a root-mean-square accuracy of <= 4.3e-3 Hz.