Inspired by the discovery of the spin-$frac{1}{2}$ doubly charmed baryon $Xi_{cc}^{++}$ and the subsequent theoretical studies of its magnetic moments, we study the magnetic moments of its spin-$frac{3}{2}$ heavy quark spin symmetry counterparts, up to the next-to-leading order in covariant baryon chiral perturbation theory (BChPT) with the extended-on-mass-shell renormalization (EOMS) scheme. With the tree-level contributions fixed by the quark model while the two low energy constants (LECs) $C$ and $H$ controlling the loop contributions determined in two ways: the quark model (case 1) and lattice QCD simulations together with the quark model (case 2), we study the quark mass dependence of the magnetic moments and compare them with the predictions of the heavy baryon chiral perturbation theory (HB ChPT). It is shown that the difference is sizable in case 1, but not in case 2 due to the smaller LECs $C$ and $H$, similar to the case of spin-$frac{1}{2}$ doubly charmed baryons. Second, we predict the magnetic moments of the spin-$frac{3}{2}$ doubly charmed baryons and compare them with those of other approaches. The predicted magnetic moments in case 2 for the spin-$frac{3}{2}$ doubly charmed baryons are closer to those of other approaches. In addition, the large differences in case 1 and case 2 for the predicted magnetic moments may indicate the inconsistency between the quark model and the lattice QCD simulations, which should be checked by future experimental or more lattice QCD data.