An experimental investigation on the periodic and chaotic oscillations in a reflex plasma source in presence of magnetic field is presented. The experiment is conducted in a reflex plasma source, consisting of two cathodes and a ring anode. A penning type DC glow discharge in an uniform axial magnetic field is initiated in the background of argon gas. The current-voltage characteristics near the breakdown voltage show a hysteresis with two distinct discharge current regimes. The effect of magnetic field on the discharge current and floating potential oscillations is studied when the discharge is operated within this hysteresis loop. At a typical axial magnetic field, the discharge transits from high discharge current regime (beyond 4-5 mA), an oscillation free regime, to a low discharge current regime (less than 1 mA). Depending upon the discharge parameters, low discharge current regime shows either the periodic or chaotic oscillation in the frequency range of 1-50 KHz. The frequency of periodic oscillation increases with the increase in magnetic field up to 90 Gauss and with further increase in magnetic field, the periodic oscillation becomes chaotic in nature.