(Abridged) Motivated by forthcoming data from the Sloan Digital Sky Survey, we present a theoretical framework that can be used to interpret Principal Component Analysis (PCA) of disk galaxy properties. We use the formalism introduced by Mo, Mao, & White to compute the observable properties of galaxies in a number of model populations, varying assumptions about which physical parameters determine structural quantities and star formation histories. We then apply PCA to these model populations. Our baseline model assumes that halo mass, spin parameter, and formation redshift are the governing input parameters and that star formation is determined by surface density through a Schmidt law. In all cases, the first principal component is primarily a measure of the shape of the spectral energy distribution (SED), and it is usually driven by variations in the spin parameter, which influences star formation through the disk surface density. The second and (in some cases) third principal components consist mainly of ``scale parameters like luminosity, disk radius, and circular velocity. However, the detailed division of these scale parameters, the disk surface brightness, and the rotation curve slope among the principal components changes significantly from model to model. Our calculations yield predictions of principal component structure for the baseline model of disk galaxy formation, and a physical interpretation of these predictions. They also show that PCA can test the core assumptions of that model and reveal the presence of additional physical parameters that may govern observable galaxy properties.