Quantitative image reconstruction in photoacoustic tomography requires the solution of a coupled physics inverse problem involvier light transport and acoustic wave propagation. In this paper we address this issue employing the radiative transfer equation as accurate model for light transport. As main theoretical results, we derive several stability and uniqueness results for the linearized inverse problem. We consider the case of single illumination as well as the case of multiple illuminations assuming full or partial data. The numerical solution of the linearized problem is much less costly than the solution of the non-linear problem. We present numerical simulations supporting the stability results for the linearized problem and demonstrate that the linearized problem already gives accurate quantitative results.