Generalized quantum measurements identifying non-orthogonal states without ambiguity often play an indispensable role in various quantum applications. For such unambiguous state discrimination scenario, we have a finite probability of obtaining inconclusive results and minimizing the probability of inconclusive results is of particular importance. In this paper, we experimentally demonstrate an adaptive generalized measurement that can unambiguously discriminate the quaternary phase-shift-keying coherent states with a near-optimal performance. Our scheme is composed of displacement operations, single photon detections and adaptive control of the displacements dependent on a history of photon detection outcomes. Our experimental results show a clear improvement of both a probability of conclusive results and a ratio of erroneous decision caused by unavoidable experimental imperfections over conventional static generalized measurements.