We propose a magnetic multilayer layout, in which the indirect exchange coupling (IEC also known as RKKY) can be switched on and off by a slight change in temperature. We demonstrate such on/off IEC switching in a Fe/Cr/FeCr-based system and obtain thermal switching widths as small as 10--20~K, essentially in any desired temperature range, including at or just above room temperature. These results add a new dimension of tunable thermal control to IEC in magnetic nanostructures, highly technological in terms of available materials and operating physical regimes.