Non-Markovian processes are widespread in natural and human-made systems, yet explicit model- ling and analysis of such systems is underdeveloped. We consider a non-Markovian dynamic network with random link activation and deletion (RLAD) and heavy tailed Mittag-Leffler distribution for the inter-event times. We derive an analytically and computationally tractable system of Kolmogorov- like forward equations utilising the Caputo derivative for the probability of having a given number of active links in the network and solve them. Simulations for the RLAD are also studied for power-law inter-event times and we show excellent agreement with the Mittag-Leffler model. This agreement holds even when the RLAD network dynamics is coupled with the susceptible-infected-susceptible (SIS) spreading dynamics. Thus, the analytically solvable Mittag-Leffler model provides an excel- lent approximation to the case when the network dynamics is characterised by power-law distributed inter-event times. We further discuss possible generalizations of our result.