In analogy to transistors in classical electronic circuits, a quantum optical switch is an important element of quantum circuits and quantum networks. Operated at the fundamental limit where a single quantum of light or matter controls another field or material system, it may enable fascinating applications such as long-distance quantum communication, distributed quantum information processing and metrology, and the exploration of novel quantum states of matter. Here, by strongly coupling a photon to a single atom trapped in the near field of a nanoscale photonic crystal cavity, we realize a system where a single atom switches the phase of a photon, and a single photon modifies the atoms phase. We experimentally demonstrate an atom-induced optical phase shift that is nonlinear at the two-photon level, a photon number router that separates individual photons and photon pairs into different output modes, and a single-photon switch where a single gate photon controls the propagation of a subsequent probe field. These techniques pave the way towards integrated quantum nanophotonic networks involving multiple atomic nodes connected by guided light.