We theoretically investigate how each orbital and valley play a role for high thermoelectric performance of SnSe. In the hole-doped regime, two kinds of valence band valleys contribute to its transport properties: one is the valley near the U-Z line, mainly consisting of the Se-$p_z$ orbitals, and the other is the one along the $Gamma$-Y line, mainly consisting of the Se-$p_y$ orbitals. Whereas the former valley plays a major role in determining the transport properties at room temperature, the latter one also offers comparable contribution and so the band structure exhibits multi-valley character by increasing the temperature. In the electron-doped regime, the conduction band valley around the $Gamma$ point solely contributes to the thermoelectric performance, where the quasi-one-dimensional electronic structure along the $a$-axis is crucial. This study provides an important knowledge for the thermoelectric properties of SnSe, and will be useful for future search of high-performance thermoelectric materials.