Inverse reinforcement learning attempts to reconstruct the reward function in a Markov decision problem, using observations of agent actions. As already observed by Russell the problem is ill-posed, and the reward function is not identifiable, even u
nder the presence of perfect information about optimal behavior. We provide a resolution to this non-identifiability for problems with entropy regularization. For a given environment, we fully characterize the reward functions leading to a given policy and demonstrate that, given demonstrations of actions for the same reward under two distinct discount factors, or under sufficiently different environments, the unobserved reward can be recovered up to a constant. Through a simple numerical experiment, we demonstrate the accurate reconstruction of the reward function through our proposed resolution.
Machine learning models are increasingly used in a wide variety of financial settings. The difficulty of understanding the inner workings of these systems, combined with their wide applicability, has the potential to lead to significant new risks for
users; these risks need to be understood and quantified. In this sub-chapter, we will focus on a well studied application of machine learning techniques, to pricing and hedging of financial options. Our aim will be to highlight the various sources of risk that the introduction of machine learning emphasises or de-emphasises, and the possible risk mitigation and management strategies that are available.
In this paper, we present a generic methodology for the efficient numerical approximation of the density function of the McKean-Vlasov SDEs. The weak error analysis for the projected process motivates us to combine the iterative Multilevel Monte Carl
o method for McKean-Vlasov SDEs cite{szpruch2019} with non-interacting kernels and projection estimation of particle densities cite{belomestny2018projected}. By exploiting smoothness of the coefficients for McKean-Vlasov SDEs, in the best case scenario (i.e $C^{infty}$ for the coefficients), we obtain the complexity of order $O(epsilon^{-2}|logepsilon|^4)$ for the approximation of expectations and $O(epsilon^{-2}|logepsilon|^5)$ for density estimation.
