We report on the X-ray spectral (using XMM-Newton data) and timing behavior (using XMM-Newton and Rossi X-ray Timing Explorer [RXTE] data) of the very faint X-ray transient and black hole system Swift J1357.2-0933 during its 2011 outburst. The XMM-Newton X-ray spectrum of this source can be adequately fitted with a soft thermal component with a temperature of ~0.22 keV (using a disc model) and a hard, non-thermal component with a photon index of ~1.6 when using a simple power-law model. In addition, an edge at ~ 0.73 keV is needed likely due to interstellar absorption. During the first RXTE observation we find a 6 mHz quasi-periodic oscillation (QPO) which is not present during any of the later RXTE observations or during the XMM-Newton observation which was taken 3 days after the first RXTE observation. The nature of this QPO is not clear but it could be related to a similar QPO seen in the black hole system H 1743-322 and to the so-called 1 Hz QPO seen in the dipping neutron-star X-ray binaries (although this later identification is quite speculative). The observed QPO has similar frequencies as the optical dips seen previously in this source during its 2011 outburst but we cannot conclusively determine that they are due to the same underlying physical mechanism. Besides the QPO, we detect strong band-limited noise in the power-density spectra of the source (as calculated from both the RXTE and the XMM-Newton data) with characteristic frequencies and strengths very similar to other black hole X-ray transients when they are at low X-ray luminosities. We discuss the spectral and timing properties of the source in the context of the proposed very high inclination of this source. We conclude that all the phenomena seen from the source cannot, as yet, be straightforwardly explained neither by an edge-on configuration nor by any other inclination configuration of the orbit.