Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userReinforcement Learning in Economics and Finance
55
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they influence not only rewards, but also future states of the world. The goal of reinforcement learning is to find an optimal policy -- a mapping from the states of the world to the set of actions, in order to maximize cumulative reward, which is a long term strategy. Exploring might be sub-optimal on a short-term horizon but could lead to optimal long-term ones. Many problems of optimal control, popular in economics for more than forty years, can be expressed in the reinforcement learning framework, and recent advances in computational science, provided in particular by deep learning algorithms, can be used by economists in order to solve complex behavioral problems. In this article, we propose a state-of-the-art of reinforcement learning techniques, and present applications in economics, game theory, operation research and finance.
rate research
Read More
The popularity of deep reinforcement learning (DRL) methods in economics have been exponentially increased. DRL through a wide range of capabilities from reinforcement learning (RL) and deep learning (DL) for handling sophisticated dynamic business environments offers vast opportunities. DRL is characterized by scalability with the potential to be applied to high-dimensional problems in conjunction with noisy and nonlinear patterns of economic data. In this work, we first consider a brief review of DL, RL, and deep RL methods in diverse applications in economics providing an in-depth insight into the state of the art. Furthermore, the architecture of DRL applied to economic applications is investigated in order to highlight the complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. The survey results indicate that DRL can provide better performance and higher accuracy as compared to the traditional algorithms while facing real economic problems at the presence of risk parameters and the ever-increasing uncertainties.
Mean field games (MFG) and mean field control problems (MFC) are frameworks to study Nash equilibria or social optima in games with a continuum of agents. These problems can be used to approximate competitive or cooperative games with a large finite number of agents and have found a broad range of applications, in particular in economics. In recent years, the question of learning in MFG and MFC has garnered interest, both as a way to compute solutions and as a way to model how large populations of learners converge to an equilibrium. Of particular interest is the setting where the agents do not know the model, which leads to the development of reinforcement learning (RL) methods. After reviewing the literature on this topic, we present a two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm. The main novelty of this method is to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion. Depending on the ratio of the two learning rates, the algorithm learns either the MFG or the MFC solution. To illustrate this method, we apply it to a mean field problem of accumulated consumption in finite horizon with HARA utility function, and to a traders optimal liquidation problem.
We analyze stop and go containment policies which produces infection cycles as periods of tight lock-downs are followed by periods of falling infection rates, which then lead to a relaxation of containment measures, allowing cases to increase again until another lock-down is imposed. The policies followed by several European countries seem to fit this pattern. We show that stop and go should lead to lower medical costs than keeping infections at the midpoint between the highs and lows produced by stop and go. Increasing the upper and reducing the lower limits of a stop and go policy by the same amount would lower the average medical load. But increasing the upper and lowering the lower limit while keeping the geometric average constant would have the opposite impact. We also show that with economic costs proportional to containment, any path that brings infections back to the original level (technically a closed cycle) has the same overall economic cost.
As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own developments and to compare with existing schemes easily. Within FinRL, virtual environments are configured with stock market datasets, trading agents are trained with neural networks, and extensive backtesting is analyzed via trading performance. Moreover, it incorporates important trading constraints such as transaction cost, market liquidity and the investors degree of risk-aversion. FinRL is featured with completeness, hands-on tutorial and reproducibility that favors beginners: (i) at multiple levels of time granularity, FinRL simulates trading environments across various stock markets, including NASDAQ-100, DJIA, S&P 500, HSI, SSE 50, and CSI 300; (ii) organized in a layered architecture with modular structure, FinRL provides fine-tuned state-of-the-art DRL algorithms (DQN, DDPG, PPO, SAC, A2C, TD3, etc.), commonly-used reward functions and standard evaluation baselines to alleviate the debugging workloads and promote the reproducibility, and (iii) being highly extendable, FinRL reserves a complete set of user-import interfaces. Furthermore, we incorporated three application demonstrations, namely single stock trading, multiple stock trading, and portfolio allocation. The FinRL library will be available on Github at link https://github.com/AI4Finance-LLC/FinRL-Library.
During its history, the ultimate goal of economics has been to develop similar frameworks for modeling economic behavior as invented in physics. This has not been successful, however, and current state of the process is the neoclassical framework that bases on static optimization. By using a static framework, however, we cannot model and forecast the time paths of economic quantities because for a growing firm or a firm going into bankruptcy, a positive profit maximizing flow of production does not exist. Due to these problems, we present a dynamic theory for the production of a profit-seeking firm where the adjustment may be stable or unstable. This is important, currently, because we should be able to forecast the possible future bankruptcies of firms due to the Covid-19 pandemic. By using the model, we can solve the time moment of bankruptcy of a firm as a function of several parameters. The proposed model is mathematically identical with Newtonian model of a particle moving in a resisting medium, and so the model explains the reasons that stop the motion too. The frameworks for modeling dynamic events in physics are thus applicable in economics, and we give reasons why physics is more important for the development of economics than pure mathematics. (JEL D21, O12) Keywords: Limitations of neoclassical framework, Dynamics of production, Economic force, Connections between economics and physics.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
