Light-pulse atom interferometers rely on the wave nature of matter and its manipulation with coherent laser pulses. They are used for precise gravimetry and inertial sensing as well as for accurate measurements of fundamental constants. Reaching higher precision requires longer interferometer times which are naturally encountered in microgravity environments such as drop-tower facilities, sounding rockets and dedicated satellite missions aiming at fundamental quantum physics in space. In all those cases, it is necessary to consider arbitrary trajectories and varying orientations of the interferometer set-up in non-inertial frames of reference. Here we provide a versatile representation-free description of atom interferometry entirely based on operator algebra to address this general situation. We show how to analytically determine the phase shift as well as the visibility of interferometers with an arbitrary number of pulses including the effects of local gravitational accelerations, gravity gradients, the rotation of the lasers and non-inertial frames of reference. Our method conveniently unifies previous results and facilitates the investigation of novel interferometer geometries.