We analyze the LHC prospects for measurements of the $tbar{t}$ pair produced exclusively in photon-photon or semi-exclusively in photon-Pomeron and Pomeron-Pomeron processes using protons tagged in forward proton detectors on both sides of the interaction point. These processes are interesting from the point of view of a possible measurement of the top quark mass and constraining models used in Beyond Standard Model physics. Focusing on the semi-leptonic channel, $tbar{t}rightarrow jjbl u_lbar{b}$, making use of the exclusive nature of the final state, together with the use of timing information provided by forward proton detectors, relevant exclusive and inclusive backgrounds are studied in detail for different luminosity (or pile-up) scenarios and found to be important for further considerations. While good prospects are found for observing the signal, the top quark mass measurement turns out not to be competitive with measurements in inclusive channels.