We report the fabrication and characterization of gate-defined hole quantum dots in monolayer and bilayer WSe$_2$. The devices were operated with gates above and below the WSe$_2$ layer to accumulate a hole gas, which for some devices was then selectively depleted to define the dot. Temperature dependence of conductance in the Coulomb blockade regime is consistent with transport through a single level, and excited state transport through the dots was observed at temperatures up to 10 K. For adjacent charge states of a bilayer WSe$_2$ dot, magnetic field dependence of excited state energies was used to estimate $g$-factors between 0.8 and 2.4 for different states. These devices provide a platform to evaluate valley-spin states in monolayer and bilayer WSe$_2$ for application as qubits.