Even with todays immense computational resources, climate models cannot resolve every cloud in the atmosphere or eddying swirl in the ocean. However, collectively these small-scale turbulent processes play a key role in setting Earths climate. Climate models attempt to represent unresolved scales via surrogate models known as parameterizations. These have limited fidelity and can exhibit structural deficiencies. Here we demonstrate that neural differential equations (NDEs) may be trained by highly resolved fluid-dynamical models of the scales to be parameterized and those NDEs embedded in an ocean model. They can incorporate conservation laws and are stable in time. We argue that NDEs provide a new route forward to the development of surrogate models for climate science, opening up exciting new opportunities.