In multi-messenger astronomy, rapid investigation of interesting transients is imperative. As an observatory with a 4$pi$ steradian field of view and $sim$99% uptime, the IceCube Neutrino Observatory is a unique facility to follow up transients, and to provide valuable insight for other observatories and inform their observing decisions. Since 2016, IceCube has been using low-latency data to rapidly respond to interesting astrophysical events reported by the multi-messenger observational community. Here, we describe the pipeline used to perform these follow up analyses and provide a summary of the 58 analyses performed as of July 2020. We find no significant signal in the first 58 analyses performed. The pipeline has helped inform various electromagnetic observing strategies, and has constrained neutrino emission from potential hadronic cosmic accelerators.