We present and experimentally realize a quantum algorithm for efficiently solving the following problem: given an $Ntimes N$ matrix $mathcal{M}$, an $N$-dimensional vector $textbf{emph{b}}$, and an initial vector $textbf{emph{x}}(0)$, obtain a target vector $textbf{emph{x}}(t)$ as a function of time $t$ according to the constraint $dtextbf{emph{x}}(t)/dt=mathcal{M}textbf{emph{x}}(t)+textbf{emph{b}}$. We show that our algorithm exhibits an exponential speedup over its classical counterpart in certain circumstances. In addition, we demonstrate our quantum algorithm for a $4times4$ linear differential equation using a 4-qubit nuclear magnetic resonance quantum information processor. Our algorithm provides a key technique for solving many important problems which rely on the solutions to linear differential equations.