We study the cavity mode frequencies of a Fabry-Perot cavity containing two vibrating dielectric membranes. We derive the equations for the mode resonances and provide approximate analytical solutions for them as a function of the membrane positions, which act as an excellent approximation when the relative and center-of-mass position of the two membranes are much smaller than the cavity length. With these analytical solutions, one finds that extremely large optomechanical coupling of the membrane relative motion can be achieved in the limit of highly reflective membranes when the two membranes are placed very close to a resonance of the inner cavity formed by them. We also study the cavity finesse of the system and verify that, under the conditions of large coupling, it is not appreciably affected by the presence of the two membranes. The achievable large values of the ratio between the optomechanical coupling and the cavity decay rate, $g/kappa$, make this two-membrane system the simplest promising platform for implementing cavity optomechanics in the strong coupling regime.