We present a pilot study on the origin and assembly history of the ICL for four galaxy clusters at 0.44<z<0.57 observed with the Hubble Space Telescope from the Cluster Lensing and Supernova Survey with Hubble (CLASH) sample. Using this sample of clusters we set an empirical limit on the amount of scatter in ICL surface brightness profiles of such clusters at z=0.5 and constrain the progenitor population and formation mechanism of the ICL by measuring the ICL surface brightness profile, the ICL color and color gradient, and the total ICL luminosity within 10<r<110 kpc. The observed scatter is physical, which we associate with differences in ICL assembly process, formation epoch, and/or ICL content. Using stellar population synthesis models we transform the observed colors to metallicity. For three of the four clusters we find clear negative gradients that, on average, decrease from super solar in the central regions of the BCG to sub-solar in the ICL. Such negative color/metallicity gradients can arise from tidal stripping of L* galaxies and/or the disruption of dwarf galaxies, but not major mergers with the BCG. We also find that the ICL at 110 kpc has a color comparable to m*+2 red sequence galaxies and a total luminosity between 10<r<110 kpc of 4-8 L*. This suggests that the ICL is dominated by stars liberated from galaxies with L>0.2 L* and that neither dwarf disruption nor major mergers with the BCG alone can explain the observed level of luminosity and remain consistent with either the observed evolution in the faint end slope of the luminosity function or predictions for the number of BCG major mergers since z=1. Taken together, the results of this pilot study are suggestive of a formation history for these clusters in which the ICL is built-up by the stripping of >0.2 L* galaxies, and disfavor significant contribution to the ICL by dwarf disruption or major mergers with the BCG.