The information theoretic observables entropy (a measure of disorder), excess entropy (a measure of complexity) and multi information are used to analyze ground-state spin configurations for disordered and frustrated model systems in 2D and 3D. For both model systems, ground-state spin configurations can be obtained in polynomial time via exact combinatorial optimization algorithms, which allowed us to study large systems with high numerical accuracy. Both model systems exhibit a continuous transition from an ordered to a disordered ground state as a model parameter is varied. By using the above information theoretic observables it is possible to detect changes in the spatial structure of the ground states as the critical point is approached. It is further possible to quantify the scaling behavior of the information theoretic observables in the vicinity of the critical point. For both model systems considered, the estimates of critical properties for the ground-state phase transitions are in good agreement with existing results reported in the literature.