Two-dimensional (2D) layered magnetic materials are generating a great amount of interest for the next generation of electronic devices thanks to their remarkable properties associated to spin dynamics. The recently discovered layered VI$_3$ ferromagnetic phase belongs to this family, although a full understanding of its properties is limited by an ill-defined crystallographic structure. This is not any longer true. Here, we investigate the VI$_3$ crystal structure upon cooling using both synchrotron X-ray and neutron powder diffraction and provide structural models for the two structural transitions occurring at 76 K and 32 K. Moreover, we confirm by magnetic measurements that VI$_3$ becomes ferromagnetic at 50 K and discuss the difficulty of solving its full magnetic structure by neutrons. We equally determined the magnetic properties of our recently reported LiVI$_3$ phase, which is alike the well-known CrI$_3$ ferromagnetic phase in terms of electronic and crystallographic structures and found to our surprise an antiferromagnetic behavior with a Neel temperature of 12 K. Such a finding provides extra clues for a better understanding of magnetism in these low dimension compounds. Finally, the easiness of preparing novel Li-based 2D magnetic materials by chemical/electrochemical means opens wide the opportunity to design materials with exotic properties.