In this article, we prove that a compact open set in the field $mathbb{Q}_p$ of $p$-adic numbers is a spectral set if and only if it tiles $mathbb{Q}_p$ by translation, and also if and only if it is $p$-homogeneous which is easy to check. We also characterize spectral sets in $mathbb{Z}/p^n mathbb{Z}$ ($pge 2$ prime, $nge 1$ integer) by tiling property and also by homogeneity. Moreover, we construct a class of singular spectral measures in $mathbb{Q}_p$, some of which are self-similar measures.