Blazars are among the most studied sources in high-energy astrophysics as they form the largest fraction of extragalactic gamma-ray sources and are considered prime candidates for being the counterparts of high-energy astrophysical neutrinos. Their reliable identification amid the many faint radio sources is a crucial step for multi-messenger counterpart associations. As the astronomical community prepares for the coming of a number of new facilities able to survey the non-thermal sky at unprecedented depths, from radio to gamma-rays, machine learning techniques for fast and reliable source identification are ever more relevant. The purpose of this work was to develop a deep learning architecture to identify blazar within a population of AGN based solely on non-contemporaneous spectral energy distribution information, collected from publicly available multi-frequency catalogues. This study uses an unprecedented amount of data, with SEDs for $approx 14,000$ sources collected with the Open Universe VOU-Blazars tool. It uses a convolutional long-short term memory neural network purposefully built for the problem of SED classification, which we describe in detail and validate. The network was able to distinguish blazars from other types of AGNs to a satisfying degree (achieving a ROC area under curve of $0.98$), even when trained on a reduced subset of the whole sample. This initial study does not attempt to classify blazars among their different sub-classes, or quantify the likelihood of any multi-frequency or multi-messenger association, but is presented as a step towards these more practically-oriented applications.