We describe instrumentation designed to perform millimeter-wave conductivity measurements in pulsed high magnetic fields at low temperatures. The main component of this system is an entirely non-metallic microwave resonator. The resonator utilizes periodic dielectric arrays (photonic bandgap structures) to confine the radiation, such that the resonant modes have a high Q-factor, and the system possesses sufficient sensitivity to measure small samples within the duration of a magnet pulse. As well as measuring the sample conductivity to probe orbital physics in metallic systems, this technique can detect the sample permittivity and permeability allowing measurement of spin physics in insulating systems. We demonstrate the system performance in pulsed magnetic fields with both electron paramagnetic resonance experiments and conductivity measurements of correlated electron systems.