Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userConvex Analytic Method Revisited: Further Optimality Results and Performance of Deterministic Stationary Control Policies
53
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
The convex analytic method (generalized by Borkar) has proved to be a very versatile method for the study of infinite horizon average cost optimal stochastic control problems. In this paper, we revisit the convex analytic method and make three primary contributions: (i) We present an existence result, under a near-monotone cost hypothesis, for controlled Markov models that lack weak continuity of the transition kernel but are strongly continuous in the action variable for every fixed state variable. (ii) For average cost stochastic control problems in standard Borel spaces, while existing results establish the optimality of stationary (possibly randomized) policies, few results are available on the optimality of stationary deterministic policies, and these are under rather restrictive hypotheses. We provide mild conditions under which an average cost optimal stochastic control problem admits optimal solutions that are deterministic and stationary, building upon a study of strategic measures by Feinberg. (iii) We establish conditions under which the performance under stationary deterministic policies is dense in the set of performance values under randomized stationary policies.
rate research
Read More
Many control policies used in various applications determine the input or action by solving a convex optimization problem that depends on the current state and some parameters. Common examples of such convex optimization control policies (COCPs) include the linear quadratic regulator (LQR), convex model predictive control (MPC), and convex control-Lyapunov or approximate dynamic programming (ADP) policies. These types of control policies are tuned by varying the parameters in the optimization problem, such as the LQR weights, to obtain good performance, judged by application-specific metrics. Tuning is often done by hand, or by simple methods such as a crude grid search. In this paper we propose a method to automate this process, by adjusting the parameters using an approximate gradient of the performance metric with respect to the parameters. Our method relies on recently developed methods that can efficiently evaluate the derivative of the solution of a convex optimization problem with respect to its parameters. We illustrate our method on several examples.
Hidden convex optimization is such a class of nonconvex optimization problems that can be globally solved in polynomial time via equivalent convex programming reformulations. In this paper, we focus on checking local optimality in hidden convex optimization. We first introduce a class of hidden convex optimization problems by jointing the classical nonconvex trust-region subproblem (TRS) with convex optimization (CO), and then present a comprehensive study on local optimality conditions. In order to guarantee the existence of a necessary and sufficient condition for local optimality, we need more restrictive assumptions. To our surprise, while (TRS) has at most one local non-global minimizer and (CO) has no local non-global minimizer, their joint problem could have more than one local non-global minimizer.
The use of deceptive strategies is important for an agent that attempts not to reveal his intentions in an adversarial environment. We consider a setting in which a supervisor provides a reference policy and expects an agent to follow the reference policy and perform a task. The agent may instead follow a different, deceptive policy to achieve a different task. We model the environment and the behavior of the agent with a Markov decision process, represent the tasks of the agent and the supervisor with linear temporal logic formulae, and study the synthesis of optimal deceptive policies for such agents. We also study the synthesis of optimal reference policies that prevents deceptive strategies of the agent and achieves the supervisors task with high probability. We show that the synthesis of deceptive policies has a convex optimization problem formulation, while the synthesis of reference policies requires solving a nonconvex optimization problem.
Testing for the infected cases is one of the most important mechanisms to control an epidemic. It enables to isolate the detected infected individuals, thereby limiting the disease transmission to the susceptible population. However, despite the significance of testing policies, the recent literature on the subject lacks a control-theoretic perspective. In this work, an epidemic model that incorporates the testing rate as a control input is presented. The proposed model differentiates the undetected infected from the detected infected cases, who are assumed to be removed from the disease spreading process in the population. First, the model is estimated and validated for COVID-19 data in France. Then, two testing policies are proposed, the so-called best-effort strategy for testing (BEST) and constant optimal strategy for testing (COST). The BEST policy is a suppression strategy that provides a lower bound on the testing rate such that the epidemic switches from a spreading to a non-spreading state. The COST policy is a mitigation strategy that provides an optimal value of testing rate that minimizes the peak value of the infected population when the total stockpile of tests is limited. Both testing policies are evaluated by predicting the number of active intensive care unit (ICU) cases and the cumulative number of deaths due to COVID-19.
In this paper we study second-order optimality conditions for non-convex set-constrained optimization problems. For a convex set-constrained optimization problem, it is well-known that second-order optimality conditions involve the support function of the second-order tangent set. In this paper we propose two approaches for establishing second-order optimality conditions for the non-convex case. In the first approach we extend the concept of the support function so that it is applicable to general non-convex set-constrained problems, whereas in the second approach we introduce the notion of the directional regular tangent cone and apply classical results of convex duality theory. Besides the second-order optimality conditions, the novelty of our approach lies in the systematic introduction and use, respectively, of direction
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
