Building occupant behavior drives significant differences in building energy use, even in automated buildings. Users distrust in the automation causes them to override settings. This results in responses that fail to satisfy both the occupants and/or the building automations objectives. The transition toward grid-interactive efficient buildings will make this evermore important as complex building control systems optimize not only for comfort, but also changing electricity costs. This paper presents a data-driven approach to study thermal comfort behavior dynamics which are not captured by standard steady-state comfort models such as predicted mean vote. The proposed model captures the time it takes for a user to override a thermostat setpoint change as a function of the manual setpoint change magnitude. The model was trained with the ecobee Donate Your Data dataset of 5 min. resolution data from 27,764 smart thermostats and occupancy sensors. The resulting population-level model shows that, on average, a 2{deg}F override will occur after ~30 mins. and an 8{deg}F override will occur in only ~15 mins., indicating the magnitude of discomfort as a key driver to the swiftness of an override. Such models could improve demand response programs through personalized controls.