In this paper, we propose a lower rank quaternion decomposition algorithm and apply it to color image inpainting. We introduce a concise form for the gradient of a real function in quaternion matrix variables. The optimality conditions of our quaternion least squares problem have a simple expression with this form. The convergence and convergence rate of our algorithm are established with this tool.