It has long been unclear if the small-scale magnetic structures on the neutron star (NS) surface could survive the fall-back episode. The study of the Hall cascade (Cumming, Arras and Zweibel 2004; Wareing and Hollerbach 2009) hinted that energy in small scales structures should dissipate on short timescales. Our new 2D magneto-thermal simulations suggest the opposite. For the first $sim$10 kyrs after the fall-back episode with accreted mass $10^{-3} M_odot$, the observed NS magnetic field appears dipolar, which is insensitive to the initial magnetic topology. In framework of the Ruderman & Sutherland (1975) vacuum gap model during this interval, non-thermal radiation is strongly suppressed. After this time the initial (i.e. multipolar) structure begins to re-emerge through the NS crust. We distinguish three evolutionary epochs for the re-emergence process: the growth of internal toroidal field, the advection of buried poloidal field, and slow Ohmic diffusion. The efficiency of the first two stages can be enhanced when small-scale magnetic structure is present. The efficient re-emergence of high order harmonics might significantly affect the curvature of the magnetospheric field lines in the emission zone. So, only after few $10^4$ yrs would the NS starts shining as a pulsar again, which is in correspondence with radio silence of central compact objects (CCOs). In addition, these results can explain the absence of good candidates for thermally emitting NSs with freshly re-emerged field among radio pulsars (Bogdanov, Ng and Kaspi 2014), as NSs have time to cool down, and supernova remnants can already dissipate.