Monolayer transition metal dichalcogenides (TMDs) hold great promise for future information processing applications utilizing a combination of electron spin and valley pseudospin. This unique spin system has led to observation of the valley Zeeman effect in neutral and charged excitonic resonances under applied magnetic fields. However, reported values of the trion valley Zeeman splitting remain highly inconsistent across studies. Here, we utilize high quality hBN encapsulated monolayer WSe$_2$ to enable simultaneous measurement of both intervalley and intravalley trion photoluminescence. We find the valley Zeeman splitting of each trion state to be describable only by a combination of three distinct g-factors, one arising from the exciton-like valley Zeeman effect, the other two, trion specific, g-factors associated with recoil of the excess electron. This complex picture goes significantly beyond the valley Zeeman effect reported for neutral excitons, and eliminates the ambiguity surrounding the magneto-optical response of trions in tungsten based TMD monolayers.