We present the point-coupling Hamiltonian as a model for frequency-independent linear optical devices acting on propagating optical modes described as a continua of harmonic oscillators. We formally integrate the Heisenberg equations of motion for this Hamiltonian, calculate its quantum scattering matrix, and show that an application of the quantum scattering matrix on an input state is equivalent to applying the inverse of classical scattering matrix on the annihilation operators describing the optical modes. We show how to construct the point-coupling Hamiltonian corresponding to a general linear optical device described by a classical scattering matrix, and provide examples of Hamiltonians for some commonly used linear optical devices. Finally, in order to demonstrate the practical utility of the point-coupling Hamiltonian, we use it to rigorously formulate a matrix-product-state based simulation for time-delayed feedback systems wherein the feedback is provided by a linear optical device described by a scattering matrix as opposed to a hard boundary condition (e.g. a mirror with less than unity reflectivity).