We investigated the occurrence and nature of superconductivity in single crystals of YFe$_2$Ge$_2$ grown out of Sn flux by employing x-ray diffraction, electrical resistivity, and specific heat measurements. We found that the residual resistivity ratio (RRR) of single crystals can be greatly improved, reaching as high as $sim$60, by decanting the crystals from the molten Sn at $sim$350$^circ$C and/or by annealing at temperatures between 550$^circ$C and 600$^circ$C. We found that samples with RRR $gtrsim$ 34 showed resistive signatures of superconductivity with the onset of the superconducting transition $T_capprox1.4$ K. RRR values vary between 35 and 65 with, on average, no systematic change in $T_c$ value, indicating that systematic changes in RRR do not lead to comparable changes in $T_c$. Specific heat measurements on samples that showed clear resistive signatures of a superconducting transition did not show any signature of a superconducting phase transition, which suggests that the superconductivity observed in this compound is either some sort of filamentary, strain stabilized superconductivity associated with small amounts of stressed YFe$_2$Ge$_2$ (perhaps at twin boundaries or dislocations) or is a second crystallographic phase present at levels below detection capability of conventional powder x-ray techniques.