An average instantaneous cross-correlation function is introduced to quantify the interaction of the financial market of a specific time. Based on the daily data of the American and Chinese stock markets, memory effect of the average instantaneous cross-correlations is investigated over different price return time intervals. Long-range time-correlations are revealed, and are found to persist up to a month-order magnitude of the price return time interval. Multifractal nature is investigated by a multifractal detrended fluctuation analysis.