The robustness of quantum control in the presence of uncertainties is important for practical applications but their quantum nature poses many challenges for traditional robust control. In addition to uncertainties in the system and control Hamiltonians and initial state preparation, there is uncertainty about interactions with the environment leading to decoherence. This paper investigates the robust performance of control schemes for open quantum systems subject to such uncertainties. A general formalism is developed, where performance is measured based on the transmission of a dynamic perturbation or initial state preparation error to a final density operator error. This formulation makes it possible to apply tools from classical robust control, especially structured singular value analysis, to assess robust performance of controlled, open quantum systems. However, there are additional difficulties that must be overcome, especially at low frequency ($sapprox0$). For example, at $s=0$, the Bloch equations for the density operator are singular, and this causes lack of continuity of the structured singular value. We address this issue by analyzing the dynamics on invariant subspaces and defining a pseudo-inverse that enables us to formulate a specialized version of the matrix inversion lemma. The concepts are demonstrated with an example of two qubits in a leaky cavity under laser driving fields and spontaneous emission. In addition, a new performance index is introduced for this system. Instead of the tracking or transfer fidelity error, performance is measured by the steady-steady entanglement generated, which is quantified by a non-linear function of the system state called concurrence. Simulations show that there is no conflict between this performance index, its log-sensitivity and stability margin under decoherence, unlike for conventional control problems [...].