We investigate a motion of a colloid in a harmonic trap driven out of equilibrium by an external non-conservative force producing a torque in the presence of a uniform magnetic field. We find that steady state exists only for a proper range of parameters such as mass, viscosity coefficient, and stiffness of the harmonic potential, and the magnetic field, which is not observed in the overdamped limit. We derive the existence condition for the steady state. We examine the combined influence of the non-conservative force and the magnetic field on non-equilibrium characteristics such as non-Boltzmann steady-state probability distribution function, probability currents, entropy production, position-velocity correlation, and violation of fluctuation-dissipation relation.