The evolution of quantum light through linear optical devices can be described by the scattering matrix $S$ of the system. For linear optical systems with $m$ possible modes, the evolution of $n$ input photons is given by a unitary matrix $U=varphi_{m,M}(S)$ given by a known homomorphism, $varphi_{m,M}$, which depends on the size of the resulting Hilbert space of the possible photon states, $M$. We present a method to decide whether a given unitary evolution $U$ for $n$ photons in $m$ modes can be achieved with linear optics or not and the inverse transformation $varphi_{m,M}^{-1}$ when the transformation can be implemented. Together with previous results, the method can be used to find a simple optical system which implements any quantum operation within the reach of linear optics. The results come from studying the adjoint map bewtween the Lie algebras corresponding to the Lie groups of the relevant unitary matrices.
Download