Extremely large magnetoresistance (XMR) was recently discovered in many non-magnetic materials, while its underlying mechanism remains poorly understood due to the complex electronic structure of these materials. Here, we report an investigation of the $alpha$-phase WP$_2$, a topologically trivial semimetal with monoclinic crystal structure (C2/m), which contrasts to the recently discovered robust type-II Weyl semimetal phase in $beta$-WP$_2$. We found that $alpha$-WP$_2$ exhibits almost all the characteristics of XMR materials: the near-quadratic field dependence of MR, a field-induced up-turn in resistivity following by a plateau at low temperature, which can be understood by the compensation effect, and high mobility of carriers confirmed by our Hall effect measurements. It was also found that the normalized MRs under different magnetic fields has the same temperature dependence in $alpha$-WP$_2$, the Kohler scaling law can describe the MR data in a wide temperature range, and there is no obvious change in the anisotropic parameter $gamma$ value with temperature. The resistance polar diagram has a peanut shape when field is rotated in $textit{ac}$ plane, which can be understood by the anisotropy of Fermi surface. These results indicate that both field-induced-gap and temperature-induced Lifshitz transition are not the origin of up-turn in resistivity in the $alpha$-WP$_2$ semimetal. Our findings establish $alpha$-WP$_2$ as a new reference material for exploring the XMR phenomena.