We have studied the dephasing of a superconducting flux-qubit coupled to a DC-SQUID based oscillator. By varying the bias conditions of both circuits we were able to tune their effective coupling strength. This allowed us to measure the effect of such a controllable and well-characterized environment on the qubit coherence. We can quantitatively account for our data with a simple model in which thermal fluctuations of the photon number in the oscillator are the limiting factor. In particular, we observe a strong reduction of the dephasing rate whenever the coupling is tuned to zero. At the optimal point we find a large spin-echo decay time of $4 mu s$.