Vortex structures in the Suns chromosphere are believed to channel energy between different layers of the solar atmosphere. We investigate the nature and dynamics of two small-scale quiet-Sun rotating structures in the chromosphere. We analyse two chromospheric structures that show clear rotational patterns in spectropolarimetric observations taken with the Interferometric Bidimensional Spectrometer (IBIS) at the Ca II 8542 AA~ line. We present the detection of spectropolarimetric signals that manifest the magnetic nature of rotating structures in the chromosphere. Our observations show two long-lived structures of plasma that each rotate clockwise inside a 10 arcsec$^{2}$~ quiet-Sun region. Their circular polarization signals are 5-10 times above the noise level. Line-of-sight Doppler velocity and horizontal velocity maps from the observations reveal clear plasma flows at and around the two structures. An MHD simulation shows these two structures are plausibly magnetically connected. Wave analysis suggests that the observed rotational vortex pattern could be due to a combination of slow actual rotation and a faster azimuthal phase speed pattern of a magneto-acoustic mode. Our results imply that the vortex structures observed in the Suns chromosphere are magnetic in nature and that they can be connected locally through the chromosphere.