The stability of asymptotic profiles of solutions to the Cauchy-Dirichlet problem for Fast Diffusion Equation (FDE, for short) is discussed. The main result of the present paper is the stability of any asymptotic profiles of least energy. It is noteworthy that this result can cover non-isolated profiles, e.g., those for thin annular domain cases. The method of proof is based on the Lojasiewicz-Simon inequality, which is usually used to prove the convergence of solutions to prescribed limits, as well as a uniform extinction estimate for solutions to FDE. Besides, local minimizers of an energy functional associated with this issue are characterized. Furthermore, the instability of positive radial asymptotic profiles in thin annular domains is also proved by applying the Lojasiewicz-Simon inequality in a different way.