In photonic neural network a key building block is the perceptron. Here, we describe and demonstrate a complex-valued photonic perceptron that combines time and space multiplexing in a fully passive silicon photonics integrated circuit. An input time dependent bit sequence is broadcasted into a few delay lines where the relative phases are trained by particle swarm algorithms toward the given task. Since only the phases of the propagating optical modes are trained, signal attenuation in the perceptron due to amplitude modulation is avoided. The perceptron performs binary pattern recognition and few bit delayed XOR operations up to 16 Gbps (limited by the used electronics) with Bit Error Rates as low as $10^{-6}$. The perceptron is fully integrated, silicon based, scalable, and can be used as a building block in large neural networks.