Charged excitons, or X$^{pm}$-trions, in monolayer transition metal dichalcogenides have binding energies of several tens of meV. Together with the neutral exciton X$^0$ they dominate the emission spectrum at low and elevated temperatures. We use charge tunable devices based on WSe$_2$ monolayers encapsulated in hexagonal boron nitride, to investigate the difference in binding energy between X$^+$ and X$^-$ and the X$^-$ fine structure. We find in the charge neutral regime, the X$^0$ emission accompanied at lower energy by a strong peak close to the longitudinal optical (LO) phonon energy. This peak is absent in reflectivity measurements, where only the X$^0$ and an excited state of the X$^0$ are visible. In the $n$-doped regime, we find a closer correspondence between emission and reflectivity as the trion transition with a well-resolved fine-structure splitting of 6~meV for X$^-$ is observed. We present a symmetry analysis of the different X$^+$ and X$^-$ trion states and results of the binding energy calculations. We compare the trion binding energy for the $n$-and $p$-doped regimes with our model calculations for low carrier concentrations. We demonstrate that the splitting between the X$^+$ and X$^-$ trions as well as the fine structure of the X$^-$ state can be related to the short-range Coulomb exchange interaction between the charge carriers.