A major strategy to prevent the spread of COVID-19 is through the limiting of in-person contacts. However, for the many disabled people who live in the community and require caregivers to assist them with activities of daily living, limiting contacts is impractical or impossible. We seek to determine which interventions can prevent infections among disabled people and their caregivers. To accomplish this, we simulate COVID-19 transmission with a compartmental model on a network. The networks incorporate heterogeneity in the risks of different types of interactions, time-dependent lockdown and reopening measures, and interaction distributions for four different groups (caregivers, disabled people, essential workers, and the general population). Among these groups, we find the probability of becoming infected is highest for caregivers and second highest for disabled people. Our analysis of the network structure illustrates that caregivers have the largest modal eigenvector centrality among the four groups. We find that two interventions -- contact-limiting by all groups and mask-wearing by disabled people and caregivers -- particularly reduce cases among disabled people and caregivers. We also test which group most effectively spreads COVID-19 by seeding infections in a subset of each group and then comparing the total number of infections as the disease spreads. We find that caregivers are the most effective spreaders of COVID-19. We then test where limited vaccine doses could be used most effectively and we find that vaccinating caregivers better protects disabled people than vaccinating the general population, essential workers, or the disabled population itself. Our results highlight the potential effectiveness of mask-wearing, contact-limiting throughout society, and strategic vaccination for limiting the exposure of disabled people and their caregivers to COVID-19.