The growing concentrations of the greenhouse gases CO2, CH4 and N2O (GHG) in the atmosphere are often considered as the dominant cause for the global warming during the past decades. The reported temperature data however do not display a simple correlation with the concentration changes since 1880 so that other reasons are to be considered to contribute notably. An important feature in this context is the shrinking of the polar ice caps observed in recent years. We have studied the direct effect of the loss of global sea ice since 1955 on the mean global temperature estimating the corresponding decrease of the terrestrial albedo. Using a simple 1-dimensional model the global warming of the surface is computed that is generated by the increase of GHG and the albedo change. A modest effect by the GHG of 0.08 K is calculated for the period 1880 to 1955 with a further increase by 0.18K for 1955 to 2015. A larger contribution of 0.55 +/-0.05 K is estimated for the melting of polar sea ice (MSI) in the latter period, i.e. it notably exceeds that of the GHG and may be compared with the observed global temperature rise of 1.0 +/- 0.1 K during the past 60 years. Our data also suggest a delayed response of the mean global temperature to the loss of sea ice with a time constant of approximately 20 years. The validity of the theoretical model and the interrelation between GHG-warming and MSI-effect are discussed.