We present a first proof-of-principle study for using deep neural networks (DNNs) as a novel search method for continuous gravitational waves (CWs) from unknown spinning neutron stars. The sensitivity of current wide-parameter-space CW searches is limited by the available computing power, which makes neural networks an interesting alternative to investigate, as they are extremely fast once trained and have recently been shown to rival the sensitivity of matched filtering for black-hole merger signals. We train a convolutional neural network with residual (short-cut) connections and compare its detection power to that of a fully-coherent matched-filtering search using the WEAVE pipeline. As test benchmarks we consider two types of all-sky searches over the frequency range from $20,mathrm{Hz}$ to $1000,mathrm{Hz}$: an `easy search using $T=10^5,mathrm{s}$ of data, and a `harder search using $T=10^6,mathrm{s}$. Detection probability $p_mathrm{det}$ is measured on a signal population for which matched filtering achieves $p_mathrm{det}=90%$ in Gaussian noise. In the easiest test case ($T=10^5,mathrm{s}$ at $20,mathrm{Hz}$) the DNN achieves $p_mathrm{det}sim88%$, corresponding to a loss in sensitivity depth of $sim5%$ versus coherent matched filtering. However, at higher-frequencies and longer observation time the DNN detection power decreases, until $p_mathrm{det}sim13%$ and a loss of $sim 66%$ in sensitivity depth in the hardest case ($T=10^6,mathrm{s}$ at $1000,mathrm{Hz}$). We study the DNN generalization ability by testing on signals of different frequencies, spindowns and signal strengths than they were trained on. We observe excellent generalization: only five networks, each trained at a different frequency, would be able to cover the whole frequency range of the search.