Our goal here is to see the space of matrices of a given size from a geometric and topological perspective, with emphasis on the families of various ranks and how they fit together. We pay special attention to the nearest orthogonal neighbor and nearest singular neighbor of a given matrix, both of which play central roles in matrix decompositions, and then against this visual backdrop examine the polar and singular value decompositions and some of their applications.