It is well known that aligned, aspherical dust grains emit polarized radiation and that the degree of polarization depends on the angle $psi$ between the interstellar magnetic field and the line of sight. However, anisotropy of the dust absorption cross sections also modulates the $total intensity$ of the radiation as the viewing geometry changes. We report a detection of this effect in the high Galactic latitude $Planck$ data, finding that the 353 GHz dust intensity per $N_{rm HI}$ is smaller when the Galactic magnetic field is mostly in the plane of the sky and larger when the field is mostly along the line of sight. These variations are of opposite sign and roughly equal magnitude as the changes in polarized intensity per $N_{rm HI}$ with $psi$, as predicted. In principle, the variation in intensity can be used in conjunction with the dust polarization angle to constrain the full 3D orientation of the Galactic magnetic field.