The parallelism of optics and the miniaturization of optical components using nanophotonic structures, such as metasurfaces present a compelling alternative to electronic implementations of convolutional neural networks. The lack of a low-power optical nonlinearity, however, requires slow and energy-inefficient