A search is performed for $D^{(*)+}_{sJ}$ mesons in the reactions $pp to D^{*+} K^0_{rm S} X$ and $pp to D^{*0} K^+ X$ using data collected at centre-of-mass energies of 7 and 8 TeV with the LHCb detector. For the $D^{*+} K^0_{rm S}$ final state, the decays $D^{*+} to D^0 pi^+$ with $D^0 to K^- pi^+$ and $D^0 to K^- pi^+ pi^+ pi^-$ are used. For $D^{*0} K^+$, the decay $D^{*0} to D^0 pi^0$ with $D^0 to K^- pi^+$ is used. A prominent $D_{s1}(2536)^+$ signal is observed in both $D^{*+} K^0_{rm S}$ and $D^{*0} K^+$ final states. The resonances $D^*_{s1}(2700)^+$ and $D^*_{s3}(2860)^+$ are also observed, yielding information on their properties, including spin-parity assignments. The decay $D^*_{s2}(2573)^+ to D^{*+} K^0_{rm S}$ is observed for the first time, at a significance of 6.9 $sigma$, and its branching fraction relative to the $D^*_{s2}(2573)^+ to D^+ K^0_{rm S}$ decay mode is measured.