-
Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra
Authors:
Darioush Kevian,
Usman Syed,
Xingang Guo,
Aaron Havens,
Geir Dullerud,
Peter Seiler,
Lianhui Qin,
Bin Hu
Abstract:
In this paper, we explore the capabilities of state-of-the-art large language models (LLMs) such as GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra in solving undergraduate-level control problems. Controls provides an interesting case study for LLM reasoning due to its combination of mathematical theory and engineering design. We introduce ControlBench, a benchmark dataset tailored to reflect the bread…
▽ More
In this paper, we explore the capabilities of state-of-the-art large language models (LLMs) such as GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra in solving undergraduate-level control problems. Controls provides an interesting case study for LLM reasoning due to its combination of mathematical theory and engineering design. We introduce ControlBench, a benchmark dataset tailored to reflect the breadth, depth, and complexity of classical control design. We use this dataset to study and evaluate the problem-solving abilities of these LLMs in the context of control engineering. We present evaluations conducted by a panel of human experts, providing insights into the accuracy, reasoning, and explanatory prowess of LLMs in control engineering. Our analysis reveals the strengths and limitations of each LLM in the context of classical control, and our results imply that Claude 3 Opus has become the state-of-the-art LLM for solving undergraduate control problems. Our study serves as an initial step towards the broader goal of employing artificial general intelligence in control engineering.
△ Less
Submitted 4 April, 2024;
originally announced April 2024.
-
Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control
Authors:
Aaron Havens,
Darioush Keivan,
Peter Seiler,
Geir Dullerud,
Bin Hu
Abstract:
Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear sy…
▽ More
Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem whose goal is to find the worst-case initial condition which shifts the terminal state the most. Then we present two PGD-based iterative methods which can be used to solve the resultant constrained maximization problem. Our analysis is not based on Lyapunov theory, and hence requires minimum information of the problem structures. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting (which is more relevant to ROA analysis of perception-based control), we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. We demonstrate the scalability and generality of our analysis tool on several numerical examples with large-scale NN policies and high-dimensional image observations. We believe that our proposed analysis serves as a meaningful initial step toward further understanding of closed-loop stability of large-scale nonlinear systems and perception-based control.
△ Less
Submitted 3 January, 2022;
originally announced January 2022.
-
Model-Free $μ$ Synthesis via Adversarial Reinforcement Learning
Authors:
Darioush Keivan,
Aaron Havens,
Peter Seiler,
Geir Dullerud,
Bin Hu
Abstract:
Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this paper, we examine the effectiveness of policy-based RL methods on an important robust control problem, namely $μ$ synthesis. We build a connection between robust adversarial RL and…
▽ More
Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this paper, we examine the effectiveness of policy-based RL methods on an important robust control problem, namely $μ$ synthesis. We build a connection between robust adversarial RL and $μ$ synthesis, and develop a model-free version of the well-known $DK$-iteration for solving state-feedback $μ$ synthesis with static $D$-scaling. In the proposed algorithm, the $K$ step mimics the classical central path algorithm via incorporating a recently-developed double-loop adversarial RL method as a subroutine, and the $D$ step is based on model-free finite difference approximation. Extensive numerical study is also presented to demonstrate the utility of our proposed model-free algorithm. Our study sheds new light on the connections between adversarial RL and robust control.
△ Less
Submitted 8 June, 2022; v1 submitted 30 November, 2021;
originally announced November 2021.
-
On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions
Authors:
Aaron Havens,
Bin Hu
Abstract:
When applying imitation learning techniques to fit a policy from expert demonstrations, one can take advantage of prior stability/robustness assumptions on the expert's policy and incorporate such control-theoretic prior knowledge explicitly into the learning process. In this paper, we formulate the imitation learning of linear policies as a constrained optimization problem, and present efficient…
▽ More
When applying imitation learning techniques to fit a policy from expert demonstrations, one can take advantage of prior stability/robustness assumptions on the expert's policy and incorporate such control-theoretic prior knowledge explicitly into the learning process. In this paper, we formulate the imitation learning of linear policies as a constrained optimization problem, and present efficient methods which can be used to enforce stability and robustness constraints during the learning processes. Specifically, we show that one can guarantee the closed-loop stability and robustness by posing linear matrix inequality (LMI) constraints on the fitted policy. Then both the projected gradient descent method and the alternating direction method of multipliers (ADMM) method can be applied to solve the resulting constrained policy fitting problem. Finally, we provide numerical results to demonstrate the effectiveness of our methods in producing linear polices with various stability and robustness guarantees.
△ Less
Submitted 23 March, 2021;
originally announced March 2021.
-
Spaces of knots in the solid torus, knots in the thickened torus, and links in the 3-sphere
Authors:
Andrew Havens,
Robin Koytcheff
Abstract:
We recursively determine the homotopy type of the space of any irreducible framed link in the 3-sphere, modulo rotations. This leads us to the homotopy type of the space of any knot in the solid torus, thus answering a question posed by Arnold. We similarly study spaces of unframed links in the 3-sphere, modulo rotations, and spaces of knots in the thickened torus. The subgroup of meridional rotat…
▽ More
We recursively determine the homotopy type of the space of any irreducible framed link in the 3-sphere, modulo rotations. This leads us to the homotopy type of the space of any knot in the solid torus, thus answering a question posed by Arnold. We similarly study spaces of unframed links in the 3-sphere, modulo rotations, and spaces of knots in the thickened torus. The subgroup of meridional rotations splits as a direct factor of the fundamental group of the space of any framed link except the unknot. Its generators can be viewed as generalizations of the Gramain loop in the space of long knots. Taking the quotient by certain such rotations relates the spaces we study. All of our results generalize previous work of Hatcher and Budney. We provide many examples and explicitly describe generators of fundamental groups.
△ Less
Submitted 5 June, 2021; v1 submitted 7 August, 2020;
originally announced August 2020.
