We report scanning tunnelling microscopy and spectroscopy (STM/STS) studies on one and two unit cell (UC) high FeSe thin films grown on Bi$_2$Se$_3$(0001). In our thin films, we find the tetragonal phase of FeSe and dumb-bell shaped defects oriented along Se-Se bond directions. In addition, we observe striped moire patterns with a periodicity of ($7.3pm 0.1$) nm generated by the mismatch between the FeSe lattice and the Bi$_2$Se$_3$ lattice. We could not find any signature of a superconducting gap in the tunneling spectra measured on the surface of one and two UC thick islands of FeSe down to 6.5 K. The spectra rather show an asymmetric behavior across and a finite density of states at the Fermi level ($E_F$) resembling those taken in the normal state of bulk FeSe.