We investigate the band structure, nematic state and superconducting gap structure of two selected FeSe single crystals containing different amount of disorder. Transport and angle-resolved photoemission spectroscopy measurements show that the small amount of disorder has little effect to the band structure and the nematic state of FeSe. However, temperature and magnetic field dependencies of specific heat for the two samples are quite different. Wave-vector-dependent gap structure are obtained from the three dimensional field-angle-resolved specific heat measurements. A small gap with two vertical-line nodes or gap minima along the $k_z$ direction is found only in the sample with higher quality. Such symmetry-unprotected nodes or gap minima are found to be smeared out by small amount of disorder, and the gap becomes isotropic in the sample of lower quality. Our study reveals that the reported controversy on the gap structure of FeSe is due to the disorder-sensitive node-like small gap.