The motivation for a cosmic muon veto (CMV) detector is to explore the possibility of locating the proposed large Iron Calorimeter (ICAL) detector at the India based Neutrino Observatory (INO) at a shallow depth. An initial effort in that direction, through the assembly and testing of a $sim$ 1 m $times$ 1 m $times$ 0.3 m plastic scintillator based detector, is described. The plan for making a CMV detector for a smaller prototype mini-ICAL is also outlined.