Recent advances in focused ion beam technology have enabled high-resolution, direct-write nanofabrication using light ions. Studies with light ions to date have, however, focused on milling of materials where sub-surface ion beam damage does not inhibit device performance. Here we report on direct-write milling of single crystal diamond using a focused beam of oxygen ions. Material quality is assessed by Raman and luminescence analysis, and reveals that the damage layer generated by oxygen ions can be removed by nonintrusive post-processing methods such as localised electron beam induced chemical etching.