A new method to generate short wavelength Free Electron Laser output with modulated polarisation at attosecond timescales is presented. Simulations demonstrate polarisation switching timescales that are four orders of magnitude faster than the current state of the art and, at X-Ray wavelengths, approaching the atomic unit of time of approximately $24$~attoseconds. Such polarisation control has significant potential in the study of ultra-fast atomic and molecular processes. The output alternates between either orthogonal linear or circularly polarised light without the need for any polarising optical elements. This facilitates operation at the high brightness X-ray wavelengths associated with FELs. As the method uses an afterburner configuration it would be relatively easy to install at exciting FEL facilities, greatly expanding their research capability.