It is well known that the efficiency of a good thermoelectric material should be optimized with respect to doping concentration. However, much less attention has been paid to the optimization of the dopants energy level. Thermoelectric materials doped with shallow levels may experience a dramatic reduction in their figures of merit at high temperatures due to the excitation of minority carriers that reduces the Seebeck coefficient and increases bipolar heat conduction. Doping with deep level impurities can delay the excitation of minority carriers as it requires a higher temperature to ionize all dopants. We find through modeling that, depending on the material type and temperature range of operation, different impurity levels (shallow or deep) will be desired to optimize the efficiency of a thermoelectric material. For different materials, we further clarify where the most preferable position of the impurity level within the band gap falls. Our research provides insights in choosing the most appropriate dopants for a thermoelectric material in order to maximize the device efficiency.