We compare ten transport codes for a system confined in a box, aiming at improved handling of the production of $Delta$ resonances and pions, which is indispensable for constraining high-density symmetry energy from observables such as the $pi^-/pi^+$ yield ratio in heavy-ion collisions. The system in a box is initialized with nucleons at saturation density and at 60 MeV temperature. The reactions $NNleftrightarrow NDelta$ and $Deltaleftrightarrow Npi$ are implemented, but the Pauli blocking and the mean-field potential are deactivated in the present comparison. Results are compared to those from the two reference cases of a chemically equilibrated ideal gas mixture and of the rate equation. In the results of the numbers of $Delta$ and $pi$, deviations from the reference values are observed in many codes, and they depend significantly on the size of the time step. These deviations are tied to different ways in ordering the sequence of collisions and decays, that take place in the same time step. Better agreements are seen in the reaction rates and the number ratios among the isospin species of $Delta$ and $pi$. These are, however, affected by the correlations, which are absent in the Boltzmann equation, but are induced by the way particle scatterings are treated in transport calculations. The uncertainty in the transport-code predictions of the $pi^-/pi^+$ ratio for the system initialized at n/p = 1.5, after letting the existing $Delta$ resonances decay, is found to be within a few percent, which is sufficiently small so that it does not strongly impact constraining the high-density symmetry energy from heavy-ion collisions. Most of the sources of uncertainties have been understood, and individual codes may be further improved. This investigation will be extended in the future to heavy-ion collisions to ensure the problems identified here remain under control.