We consider the Jaynes-Cummings model of a single quantum spin $s$ coupled to a harmonic oscillator in a parameter regime where the underlying classical dynamics exhibits an unstable equilibrium point. This state of the model is relevant to the physics of cold atom systems, in non-equilibrium situations obtained by fast sweeping through a Feshbach resonance. We show that in this integrable system with two degrees of freedom, for any initial condition close to the unstable point, the classical dynamics is controlled by a singularity of the focus-focus type. In particular, it displays the expected monodromy, which forbids the existence of global action-angle coordinates. Explicit calculations of the joint spectrum of conserved quantities reveal the monodromy at the quantum level, as a dislocation in the lattice of eigenvalues. We perform a detailed semi-classical analysis of the associated eigenstates. Whereas most of the levels are well described by the usual Bohr-Sommerfeld quantization rules, properly adapted to polar coordinates, we show how these rules are modified in the vicinity of the critical level. The spectral decomposition of the classically unstable state is computed, and is found to be dominated by the critical WKB states. This provides a useful tool to analyze the quantum dynamics starting from this particular state, which exhibits an aperiodic sequence of solitonic pulses with a rather well defined characteristic frequency.