We have identified 335 galaxy cluster and group candidates, 106 of which are at z > 1, using a 4.5 um selected sample of objects from a 7.25 deg^2 region in the Spitzer Infrared Array Camera (IRAC) Shallow Survey. Clusters were identified as 3-dimensional overdensities using a wavelet algorithm, based on photometric redshift probability distributions derived from IRAC and NOAO Deep Wide-Field Survey data. We estimate only ~10% of the detections are spurious. To date 12 of the z > 1 candidates have been confirmed spectroscopically, at redshifts from 1.06 to 1.41. Velocity dispersions of ~750 km/s for two of these argue for total cluster masses well above 10^14 M_sun, as does the mass estimated from the rest frame near infrared stellar luminosity. Although not selected to contain a red sequence, some evidence for red sequences is present in the spectroscopically confirmed clusters, and brighter galaxies are systematically redder than the mean galaxy color in clusters at all redshifts. The mean I - [3.6] color for cluster galaxies up to z ~ 1 is well matched by a passively evolving model in which stars are formed in a 0.1 Gyr burst starting at redshift z_f = 3. At z > 1, a wider range of formation histories is needed, but higher formation redshifts (i.e. z_f > 3) are favored for most clusters.