The magnetization and magnetic torque of a high-quality single crystal of Sr$_2$RuO$_4$ have been measured down to 0.1 K under a precise control of the magnetic-field orientation. When the magnetic field is applied exactly parallel to the $ab$ plane, a sharp magnetization jump $4pidelta M$ of $(0.74 pm 0.15)$ G at the upper critical field $H_{{rm c2},{ab}} sim 15$ kOe with a field hysteresis of 100 Oe is observed at low temperatures, evidencing a first-order superconducting-normal transition. A strong magnetic torque appearing when $H$ is slightly tilted away from the $ab$ plane confirms an intrinsic anisotropy $varGamma=xi_a/xi_c$ of as large as 60 even at 100 mK, in contrast with the observed $H_{{rm c2}}$ anisotropy of $sim 20$. The present results raise fundamental issues in both the existing spin-triplet and spin-singlet scenarios, providing, in turn, crucial hints toward the resolution of the superconducting nature of Sr$_2$RuO$_4$.