The flow of in-plane current through ultrathin magnetic heterostructures can cause magnetization switching or domain wall nucleation owing to bulk and interfacial effects. Within the magnetic layer, the current can create magnetic instabilities via spin transfer torques (STT). At interface(s), spin current generated from the spin Hall effect in a neighboring layer can exert torques, referred to as the spin Hall torques, on the magnetic moments. Here, we study current induced magnetization switching in perpendicularly magnetized CoFeB/MgO heterostructures with a heavy metal (HM) underlayer. Depending on the thickness of the HM underlayer, we find distinct differences in the inplane field dependence of the threshold switching current. The STT is likely responsible for the magnetization reversal for the thinner underlayer films whereas the spin Hall torques cause the switching for thicker underlayer films. For the latter, we find differences in the switching current for positive and negative currents and initial magnetization directions. We find that the growth process during the film deposition introduces an anisotropy that breaks the symmetry of the system and causes the asymmetric switching. The presence of such symmetry breaking anisotropy enables deterministic magnetization switching at zero external fields.