Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userChance-Constrained Trajectory Optimization for Non-linear Systems with Unknown Stochastic Dynamics
110
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Iterative trajectory optimization techniques for non-linear dynamical systems are among the most powerful and sample-efficient methods of model-based reinforcement learning and approximate optimal control. By leveraging time-variant local linear-quadratic approximations of system dynamics and reward, such methods can find both a target-optimal trajectory and time-variant optimal feedback controllers. However, the local linear-quadratic assumptions are a major source of optimization bias that leads to catastrophic greedy updates, raising the issue of proper regularization. Moreover, the approximate models disregard for any physical state-action limits of the system causes further aggravation of the problem, as the optimization moves towards unreachable areas of the state-action space. In this paper, we address the issue of constrained systems in the scenario of online-fitted stochastic linear dynamics. We propose modeling state and action physical limits as probabilistic chance constraints linear in both state and action and introduce a new trajectory optimization technique that integrates these probabilistic constraints by optimizing a relaxed quadratic program. Our empirical evaluations show a significant improvement in learning robustness, which enables our approach to perform more effective updates and avoid premature convergence observed in state-of-the-art algorithms.
rate research
Read More
Continued great efforts have been dedicated towards high-quality trajectory generation based on optimization methods, however, most of them do not suitably and effectively consider the situation with moving obstacles; and more particularly, the future position of these moving obstacles in the presence of uncertainty within some possible prescribed prediction horizon. To cater to this rather major shortcoming, this work shows how a variational Bayesian Gaussian mixture model (vBGMM) framework can be employed to predict the future trajectory of moving obstacles; and then with this methodology, a trajectory generation framework is proposed which will efficiently and effectively address trajectory generation in the presence of moving obstacles, and also incorporating presence of uncertainty within a prediction horizon. In this work, the full predictive conditional probability density function (PDF) with mean and covariance is obtained, and thus a future trajectory with uncertainty is formulated as a collision region represented by a confidence ellipsoid. To avoid the collision region, chance constraints are imposed to restrict the collision probability, and subsequently a nonlinear MPC problem is constructed with these chance constraints. It is shown that the proposed approach is able to predict the future position of the moving obstacles effectively; and thus based on the environmental information of the probabilistic prediction, it is also shown that the timing of collision avoidance can be earlier than the method without prediction. The tracking error and distance to obstacles of the trajectory with prediction are smaller compared with the method without prediction.
A probabilistic performance-oriented control design optimization approach is introduced for flight systems. Aiming at estimating rare-event probabilities accurately and efficiently, subset simulation is combined with surrogate modeling techniques to improve efficiency. At each level of subset simulation, the samples that are close to the failure domain are employed to construct a surrogate model. The existing surrogate is then refined progressively. In return, seed and sample candidates are screened by the updated surrogate, thus saving a large number of calls to the true model and reducing the computational expense. Afterwards, control parameters are optimized under rare-event chance constraints to directly guarantee system performance. Simulations are conducted on an aircraft longitudinal model subject to parametric uncertainties to demonstrate the efficiency and accuracy of this method.
This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this paper we first show that the steady-state transfer function of a linear system can be computed directly from control experiments, bypassing explicit model identification. Then, we leverage the estimated transfer function to design a controller -- which is inspired by stochastic gradient descent methods -- that regulates the system to the solution of the prescribed optimization problem. A distinguishing feature of our methods is that they do not require any knowledge of the system dynamics, disturbance terms, or their distributions. Our technical analysis combines concepts and tools from behavioral system theory, stochastic optimization with decision-dependent distributions, and stability analysis. We illustrate the applicability of the framework on a case study for mobility-on-demand ride service scheduling in Manhattan, NY.
In this paper, we consider a Nash equilibrium seeking problem for a class of high-order multi-agent systems with unknown dynamics. Different from existing results for single integrators, we aim to steer the outputs of this class of uncertain high-order agents to the Nash equilibrium of some noncooperative game in a distributed manner. To overcome the difficulties brought by the high-order structure, unknown nonlinearities, and the regulation requirement, we first introduce a virtual player for each agent and solve an auxiliary noncooperative game for them. Then, we develop a distributed adaptive protocol by embedding this auxiliary game dynamics into some proper tracking controller for the original agent to resolve this problem. We also discuss the parameter convergence problem under certain persistence of excitation condition. The efficacy of our algorithms is verified by numerical examples.
We study identification of linear systems with multiplicative noise from multiple trajectory data. A least-squares algorithm, based on exploratory inputs, is proposed to simultaneously estimate the parameters of the nominal system and the covariance matrix of the multiplicative noise. The algorithm does not need prior knowledge of the noise or stability of the system, but requires mild conditions of inputs and relatively small length for each trajectory. Identifiability of the noise covariance matrix is studied, showing that there exists an equivalent class of matrices that generate the same second-moment dynamic of system states. It is demonstrated how to obtain the equivalent class based on estimates of the noise covariance. Asymptotic consistency of the algorithm is verified under sufficiently exciting inputs and system controllability conditions. Non-asymptotic estimation performance is also analyzed under the assumption that system states and noise are bounded, providing vanishing high-probability bounds as the number of trajectories grows to infinity. The results are illustrated by numerical simulations.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
