We present upper critical field data for $kappa$-(BEDT-TTF)$_2$Cu(NCS)$_2$ with the magnetic field close to parallel and parallel to the conducting layers. We show that we can eliminate the effect of vortex dynamics in these layered materials if the layers are oriented within 0.3 degrees of parallel to the applied magnetic field. Eliminating vortex effects leaves one remaining feature in the data that corresponds to the Pauli paramagnetic limit ($H_p$). We propose a semi-empirical method to calculate the $H_p$ in quasi 2D superconductors. This method takes into account the energy gap of each of the quasi 2D superconductors, which is calculated from specific heat data, and the influence of many body effects. The calculated Pauli paramagnetic limits are then compared to critical field data for the title compound and other organic conductors. Many of the examined quasi 2D superconductors, including the above organic superconductors and CeCoIn$_5$, exhibit upper critical fields that exceed their calculated $H_p$ suggesting unconventional superconductivity. We show that the high field low temperature state in $kappa$-(BEDT-TTF)$_2$Cu(NCS)$_2$ is consistent with the Fulde Ferrell Larkin Ovchinnikov state.