An oriented hypergraph is a hypergraph where each vertex-edge incidence is given a label of $+1$ or $-1$. We define the adjacency, incidence and Laplacian matrices of an oriented hypergraph and study each of them. We extend several matrix results known for graphs and signed graphs to oriented hypergraphs. New matrix results that are not direct generalizations are also presented. Finally, we study a new family of matrices that contains walk information.