Superconductivity is a phenomenon where the macroscopic quantum coherence appears due to the pairing of electrons. This offers a fascinating arena to study the physics of broken gauge symmetry. However, the important symmetries in superconductors are not only the gauge invariance. Especially, the symmetry properties of the pairing, i.e., the parity and spin-singlet/spin-triplet, determine the physical properties of the superconducting state. Recently it has been recognized that there is the important third symmetry of the pair amplitude, i.e., even or odd parity with respect to the frequency. The conventional uniform superconducting states correspond to the even-frequency pairing, but the recent finding is that the odd-frequency pair amplitude arises in the spatially non-uniform situation quite ubiquitously. Especially, this is the case in the Andreev bound state (ABS) appearing at the surface/interface of the sample. The other important recent development is on the nontrivial topological aspects of superconductors. As the band insulators are classified by topological indices into (i) conventional insulator, (ii) quantum Hall insulator, and (iii) topological insulator, also are the gapped superconductors. The influence of the nontrivial topology of the bulk states appears as the edge or surface of the sample. In the superconductors, this leads to the formation of zero energy ABS (ZEABS). Therefore, the ABSs of the superconductors are the place where the symmetry and topology meet each other which offer the stage of rich physics. In this review, we discuss the physics of ABS from the viewpoint of the odd-frequency pairing, the topological bulk-edge correspondence, and the interplay of these two issues. It is described how the symmetry of the pairing and topological indices determines the absence/presence of the ZEABS, its energy dispersion, and properties as the Majorana fermions.