A new robust stochastic volatility (SV) model having Student-t marginals is proposed. Our process is defined through a linear normal regression model driven by a latent gamma process that controls temporal dependence. This gamma process is strategically chosen to enable us to find an explicit expression for the pairwise joint density function of the Student-t response process. With this at hand, we propose a composite likelihood (CL) based inference for our model, which can be straightforwardly implemented with a low computational cost. This is a remarkable feature of our Student-t SV process over existing SV models in the literature that involve computationally heavy algorithms for estimating parameters. Aiming at a precise estimation of the parameters related to the latent process, we propose a CL Expectation-Maximization algorithm and discuss a bootstrap approach to obtain standard errors. The finite-sample performance of our composite likelihood methods is assessed through Monte Carlo simulations. The methodology is motivated by an empirical application in the financial market. We analyze the relationship, across multiple time periods, between various US sector Exchange-Traded Funds returns and individual companies stock price returns based on our novel Student-t model. This relationship is further utilized in selecting optimal financial portfolios.