The PHENIX collaboration has measured high-$p_T$ dihadron correlations in $p$$+$$p$, $p$$+$Al, and $p$$+$Au collisions at $sqrt{s_{_{NN}}}=200$ GeV. The correlations arise from inter- and intra-jet correlations and thus have sensitivity to nonperturbative effects in both the initial and final states. The distributions of $p_{rm out}$, the transverse momentum component of the associated hadron perpendicular to the trigger hadron, are sensitive to initial and final state transverse momenta. These distributions are measured multi-differentially as a function of $x_E$, the longitudinal momentum fraction of the associated hadron with respect to the trigger hadron. The near-side $p_{rm out}$ widths, sensitive to fragmentation transverse momentum, show no significant broadening between $p$$+$Au, $p$$+$Al, and $p$$+$$p$. The away-side nonperturbative $p_{rm out}$ widths are found to be broadened in $p$$+$Au when compared to $p$$+$$p$; however, there is no significant broadening in $p$$+$Al compared to $p$$+$$p$ collisions. The data also suggest that the away-side $p_{rm out}$ broadening is a function of $N_{rm coll}$, the number of binary nucleon-nucleon collisions, in the interaction. The potential implications of these results with regard to initial and final state transverse momentum broadening and energy loss of partons in a nucleus, among other nuclear effects, are discussed.