Nematic superconductors are characterized by an apparent crystal symmetry breaking that results in the anisotropy of the in-plain upper critical magnetic field $H_{c2}$. The symmetry breaking is usually attributed to the strain of the crystal lattice. The nature and the value of the strain are debatable. We perform systematic measurements of the $H_{c2}$ anisotropy in the high-quality Sr$_x$Bi$_2$Se$_3$ single crystals in the temperature range 1.8~K$<T<T_capprox 2.7$~K using temperature stabilization with an accuracy of 0.0001 K. We observe that in all tested samples the anisotropy is practically constant when $T<0.8 T_c$ and smoothly decreases at higher temperatures without any sign of singularity when $Trightarrow T_c$. Such a behavior can be understood in the framework of the Ginzburg-Landau (GL) theory for the nematic superconductors assuming that the samples are appreciably deformed. The nature of the strain, the values of the GL parameters, and the effects of disorder near $T_c$ are discussed. Similar measurements can be used to study a nature of the symmetry breaking in the other nematic superconductors.