We consider the electrical current through a magnetic point contact in the limit of a strong inelastic scattering of electrons. In this limit local Joule heating of the contact region plays a decisive role in determining the transport properties of the point contact. We show that if an applied constant bias voltage exceeds a critical value, the stationary state of the system is unstable, and that periodic, non-harmonic oscillations in time of both the electrical current through the contact and the local temperature in the contact region develop spontaneously. Our estimations show that the necessary experimental conditions for observing such oscillations with characteristic frequencies in the range $10^8 div 10^9$ Hz can easily be met. We also show a possibility to manipulate upon the magnetization direction of a magnetic grain coupled through a point contact to a bulk ferromagnetic by exciting the above-mentioned thermal-electric oscillations.