The Cu$_2$BaSnS$_4$ (CBTS) and Cu$_2$SrSnS$_4$ (CSTS) semiconductors have been recently proposed as potential wide band gap photovoltaic absorbers. Although several measurements indicate that they are less affected by band tailing than their parent compound Cu$_2$ZnSnS$_4$, their photovoltaic efficiencies are still low. To identify possible issues, we characterize CBTS and CSTS in parallel by a variety of spectroscopic methods complemented by first-principles calculations. Two main problems are identified in both materials. The first is the existence of deep defect transitions in low-temperature photoluminescence, pointing to a high density of bulk recombination centers. The second is a low electron affinity, which emphasizes the need for an alternative heterojunction partner and electron contact. We also find a tendency for downward band bending at the surface of both materials. In CBTS, this effect is sufficiently large to cause carrier type inversion, which may enhance carrier separation and mitigate interface recombination. Optical absorption at room temperature is exciton-enhanced in both CBTS and CSTS. Deconvolution of excitonic effects yields band gaps that are about 100 meV higher than previous estimates based on Tauc plots. Although the two investigated materials are remarkably similar in an idealized, defect-free picture, the present work points to CBTS as a more promising absorber than CSTS for tandem photovoltaics.