The population of exoplanetary systems detected by Kepler provides opportunities to refine our understanding of planet formation. Unraveling the conditions needed to produce the observed exoplanets will sallow us to make informed predictions as to where habitable worlds exist within the galaxy. In this paper, we examine using N-body simulations how the properties of planetary systems are determined during the final stages of assembly. While accretion is a chaotic process, trends in the ensemble properties of planetary systems provide a memory of the initial distribution of solid mass around a star prior to accretion. We also use EPOS, the Exoplanet Population Observation Simulator, to account for detection biases and show that different accretion scenarios can be distinguished from observations of the Kepler systems. We show that the period of the innermost planet, the ratio of orbital periods of adjacent planets, and masses of the planets are determined by the total mass and radial distribution of embryos and planetesimals at the beginning of accretion. In general, some amount of orbital damping, either via planetesimals or gas, during accretion is needed to match the whole population of exoplanets. Surprisingly, all simulated planetary systems have planets that are similar in size, showing that the peas in a pod pattern can be consistent with both a giant impact scenario and a planet migration scenario. The inclusion of material at distances larger than what Kepler observes has a profound impact on the observed planetary architectures, and thus on the formation and delivery of volatiles to possible habitable worlds.