We investigate the delayed, secondary GeV-TeV emission of gamma-ray bursts and its potential to probe the nature of intergalactic magnetic fields. Geometrical effects are properly taken into account for the time delay between primary high energy photons and secondary inverse Compton photons from electron-positron pairs, which are produced in $gamma$-$gamma$ interactions with background radiation fields and deflected by intervening magnetic fields. The time-dependent spectra of the delayed emission are evaluated for a wide range of magnetic field strengths and redshifts. The typical flux and delay time of secondary photons from bursts at $z sim 1$ are respectively $sim 10^{-8}$ GeV cm$^{-2}$ s$^{-1}$ and $sim 10^4$ s if the field strengths are $sim 10^{-18}$ G, as might be the case in intergalactic void regions. We find crucial differences between the cases of coherent and tangled magnetic fields, as well as dependences on the field coherence length.