Recent radio observations have unveiled the existence of a number of radio pulsars with spin-down derived magnetic fields in the magnetar range. However, their observational properties appears to be more similar to classical radio pulsars than to magnetars. To shed light on this puzzle we first have to determine whether the spin-down derived magnetic field values for these radio pulsars are indeed representative of the actual neutron star magnetic field or if they are polluted, e.g. by the effects of a torque from a fallback disk. To investigate this possibility, we have performed deep IR observations of one of these high magnetic field radio pulsars (PSR J1119-6127) with the ESO VLT to search for IR emission which can be associated with a disk. No IR emission is detected from the pulsar position down to J=24, H=23, Ks=22. By comparing our flux upper limits with the predictions of fallback disk models, we have found that we can only exclude the presence of a disk with accretion rate dot M >3x10^16 g/s. This lower limit cannot rule out the presence of a substantial disk torque on the pulsar, which would then lead to overestimate the value of the magnetic field inferred from P and dot P.