The direct domain coupling of spontaneous ferroelectric polarization and net magnetic moment can result in giant magnetoelectric (ME) coupling, which is essential to achieve mutual control and practical applications of multiferroics. Recently, the possible bulk domain coupling, the mutual control of ferroelectricity (FE) and weak ferromagnetism (WFM) have been theoretically predicted in hexagonal LuFeO3. Here, we report the first successful growth of highly-cleavable Sc-stabilized hexagonal Lu0.6Sc0.4FeO3 (h-LSFO) single crystals, as well as the first visualization of their intrinsic cloverleaf pattern of vortex FE domains and large-loop WFM domains. The vortex FE domains are on the order of 0.1-1 {mu}m in size. On the other hand, the loop WFM domains are ~100 {mu}m in size, and there exists no interlocking of FE and WFM domain walls. These strongly manifest the decoupling between FE and WFM in h-LSFO. The domain decoupling can be explained as the consequence of the structure-mediated coupling between polarization and dominant in-plane antiferromagnetic spins according to the theoretical prediction, which reveals intriguing interplays between FE, WFM, and antiferromagnetic orders in h-LSFO. Our results also indicate that the magnetic topological charge tends to be identical to the structural topological charge. This could provide new insights into the induction of direct coupling between magnetism and ferroelectricity mediated by structural distortions, which will be useful for the future applications of multiferroics.