A novel version of the Continuous-Time Random Walk (CTRW) model with memory is developed. This memory means the dependence between arbitrary number of successive jumps of the process, while waiting times between jumps are considered as i.i.d. random variables. The dependence was found by analysis of empirical histograms for the stochastic process of a single share price on a market within the high frequency time scale, and justified theoretically by considering bid-ask bounce mechanism containing some delay characteristic for any double-auction market. Our model turns out to be exactly analytically solvable, which enables a direct comparison of its predictions with their empirical counterparts, for instance, with empirical velocity autocorrelation function. Thus this paper significantly extends the capabilities of the CTRW formalism.