Increasingly, information systems rely on computational, storage, and network resources deployed in third-party facilities or are supported by service providers. Such an approach further exacerbates cybersecurity concerns constantly raised by numerous incidents of security and privacy attacks resulting in data leakage and identity theft, among others. These have in turn forced the creation of stricter security and privacy related regulations and have eroded the trust in cyberspace. In particular, security related services and infrastructures such as Certificate Authorities (CAs) that provide digital certificate service and Third-Party Authorities (TPAs) that provide cryptographic key services, are critical components for establishing trust in Internet enabled applications and services. To address such trust issues, various transparency frameworks and approaches have been recently proposed in the literature. In this paper, we propose a Transparent and Trustworthy TPA using Blockchain (T3AB) to provide transparency and accountability to the trusted third-party entities, such as honest-but-curious third-party IaaS servers, and coordinators in various privacy-preserving machine learning (PPML) approaches. T3AB employs the Ethereum blockchain as the underlying public ledger and also includes a novel smart contract to automate accountability with an incentive mechanism that motivates participants to participate in auditing, and punishes unintentional or malicious behaviors. We implement T3AB, and show through experimental evaluation in the Ethereum official test network, Rinkeby, that the framework is efficient. We also formally show the security guarantee provided by T3AB, and analyze the privacy guarantee and trustworthiness it provides.