VTechWorks staff will be away for the winter holidays starting Tuesday, December 24, 2024, through Wednesday, January 1, 2025, and will not be replying to requests during this time. Thank you for your patience, and happy holidays!

Browsing by Author "Jin, Ming"

Now showing 1 - 11 of 11
  • Thumbnail Image
    Adversarial Unlearning of Backdoors via Implicit Hypergradient
    Zeng, Yi; Chen, Si; Park, Won; Mao, Morley; Jin, Ming; Jia, Ruoxi (2022)
    We propose a minimax formulation for removing backdoors from a given poisoned model based on a small set of clean data. This formulation encompasses much of prior work on backdoor removal. We propose the Implicit Bacdoor Adversarial Unlearning (I-BAU) algorithm to solve the minimax. Unlike previous work, which breaks down the minimax into separate inner and outer problems, our algorithm utilizes the implicit hypergradient to account for the interdependence between inner and outer optimization. We theoretically analyze its convergence and the generalizability of the robustness gained by solving minimax on clean data to unseen test data. In our evaluation, we compare I-BAU with six stateof- art backdoor defenses on seven backdoor attacks over two datasets and various attack settings, including the common setting where the attacker targets one class as well as important but underexplored settings where multiple classes are targeted. I-BAU’s performance is comparable to and most often significantly better than the best baseline. Particularly, its performance is more robust to the variation on triggers, attack settings, poison ratio, and clean data size. Moreover, I-BAU requires less computation to take effect; particularly, it is more than 13X faster than the most efficient baseline in the single-target attack setting. Furthermore, it can remain effective in the extreme case where the defender can only access 100 clean samples—a setting where all the baselines fail to produce acceptable results.
  • Thumbnail Image
    Control of Grid-Connected Converters using Deep Learning
    Ghidewon-Abay, Sengal (Virginia Tech, 2023-01-12)
    With the rise of inverter-based resources (IBRs) within the power system, the control of grid-connected converters (GCC) has become pertinent due to the fact they interface IBRs to the grid. The conventional method of control for grid-connected converters (GCCs) such as the voltage-sourced converter (VSC) is through a decoupled control loop in the synchronous reference frame. However, this model-based control method is sensitive to parameter changes causing deterioration in controller performance. Data-driven approaches such as machine learning can be utilized to design controllers that are capable of operating GCCs in various system conditions. This work reviews different machine learning applications in power systems as well as the conventional method of controlling a VSC. It explores a deep learning-based control method for a three-phase grid-connected VSC, specifically utilizing a long short-term memory (LSTM) network for robust control. Simulations of a conventional controlled VSC are conducted using Simulink to collect data for training the LSTM-based controller. The LSTM model is built and trained using the Keras and TensorFlow libraries in Python and tested in Simulink. The performance of the LSTM-based controller is evaluated under different case studies and compared to the conventional method of control. Simulation results demonstrate the effectiveness of this approach by outperforming the conventional controller and maintaining stability under different system parameter changes.
  • Thumbnail Image
    Derivative-Free Meta-Blackbox Optimization on Manifold
    Sel, Bilgehan (Virginia Tech, 2024-06)
    Solving a sequence of high-dimensional, nonconvex, but potentially similar optimization problems poses a significant computational challenge in various engineering applications. This thesis presents the first meta-learning framework that leverages the shared structure among sequential tasks to improve the computational efficiency and sample complexity of derivative-free optimization. Based on the observation that most practical high-dimensional functions lie on a latent low-dimensional manifold, which can be further shared among problem instances, the proposed method jointly learns the meta-initialization of a search point and a meta-manifold. This novel approach enables the efficient adaptation of the optimization process to new tasks by exploiting the learned meta-knowledge. Theoretically, the benefit of meta-learning in this challenging setting is established by proving that the proposed method achieves improved convergence rates and reduced sample complexity compared to traditional derivative-free optimization techniques. Empirically, the effectiveness of the proposed algorithm is demonstrated in two high-dimensional reinforcement learning tasks, showcasing its ability to accelerate learning and improve performance across multiple domains. Furthermore, the robustness and generalization capabilities of the meta-learning framework are explored through extensive ablation studies and sensitivity analyses. The thesis highlights the potential of meta-learning in tackling complex optimization problems and opens up new avenues for future research in this area.
  • Thumbnail Image
    Energy And Power Systems Simulated Attack Algorithm For Defense Testbed And Analysis
    Ruttle, Zachary Andrew (Virginia Tech, 2023-05-31)
    The power grid has evolved over the course of many decades with the usage of cyber systems and communications such as Supervisory Control And Data Acquisition (SCADA); however, due to their connectivity to the internet, the cyber-power system can be infiltrated by malicious attackers. Encryption is not a singular solution. Currently, there are several cyber security measures in development, including those based on artificial intelligence. However, there is a need for a varying but consistent attack algorithm to serve as a testbed for these AI or other practices to be trained and tested. This is important because in the event of a real attacker, it is not possible to know exactly where they will attack and in what order. Therefore, the proposed method in this thesis is to use criminology concepts and fuzzy logic inference to create this algorithm and determine its effectiveness in making decisions on a cyber-physical system model. The method takes various characteristics of the attacker as an input, builds their ideal target node, and then compares the nodes to the high-impact target and chooses one as the goal. Based on that target and their knowledge, the attackers will attack nodes if they have resources. The results show that the proposed method can be used to create a variety of attacks with varying damaging effects, and one other set of tests shows the possibility for multiple attacks, such as denial of service and false data injection. The proposed method has been validated using an extended cyber-physical IEEE 13-node distribution system and sensitivity tests to ensure that the ruleset created would take each of the inputs well.
  • Thumbnail Image
    Imitation Learning with Stability and Safety Guarantees
    Yin, He; Seiler, Peter; Jin, Ming; Arcak, Murat (IEEE, 2022-01-01)
    A method is presented to learn neural network (NN) controllers with stability and safety guarantees through imitation learning (IL). Convex stability and safety conditions are derived for linear time-invariant systems with NN controllers by merging Lyapunov theory with local quadratic constraints to bound the activation functions in the NN. These conditions are incorporated in the IL process, which minimizes the IL loss, and maximizes the volume of the region of attraction associated with the NN controller simultaneously. An alternating direction method of multipliers based algorithm is proposed to solve the IL problem. The method is illustrated on a vehicle lateral control example.
  • Thumbnail Image
    Learning-to-Learn to Guide Random Search: Derivative-Free Meta Blackbox Optimization on Manifold
    Sel, Bilgehan; Tawaha, Ahmad; Ding, Yuhao; Jia, Ruoxi; Ji, Bo; Lavaei, Javad; Jin, Ming (2023-01-01)
    Solving a sequence of high-dimensional, nonconvex, but potentially similar optimization problems poses a computational challenge in engineering applications. We propose the first meta-learning framework that leverages the shared structure among sequential tasks to improve the computational efficiency and sample complexity of derivative-free optimization. Based on the observation that most practical high-dimensional functions lie on a latent low-dimensional manifold, which can be further shared among instances, our method jointly learns the meta-initialization of a search point and a meta-manifold. Theoretically, we establish the benefit of meta-learning in this challenging setting. Empirically, we demonstrate the effectiveness of the proposed algorithm in two high-dimensional reinforcement learning tasks.
  • Thumbnail Image
    Machine Learning and Quantum Computing for Optimization Problems in Power Systems
    Gupta, Sarthak (Virginia Tech, 2023-01-26)
    While optimization problems are ubiquitous in all domains of engineering, they are of critical importance to power systems engineers. A safe and economical operation of the power systems entails solving many optimization problems such as security-constrained unit commitment, economic dispatch, optimal power flow, optimal planning, etc. Although traditional optimization solvers and software have been successful so far in solving these problems, there is a growing need to accelerate the solution process. This need arises on account of several aspects of grid modernization, such as distributed energy resources, renewable energy, smart inverters, batteries, etc, that increase the number of decision variables involved. Moreover, the technologies entail faster dynamics and unpredictability, further demanding a solution speedup. Yet another concern is the growing communication overhead that accompanies this large-scale, high-speed, decision-making process. This thesis explores three different directions to address such concerns. The first part of the thesis explores the learning-to-optimize paradigm whereby instead of solving the optimization problems, machine learning (ML) models such as deep neural networks (DNNs) are trained to predict the solution of the optimization problems. The second part of the thesis also employs deep learning, but in a different manner. DNNs are utilized to model the dynamics of IEEE 1547.8 standard-based local Volt/VAR control rules, and then leverage efficient deep learning libraries to solve the resulting optimization problem. The last part of the thesis dives into the evolving field of quantum computing and develops a general strategy for solving stochastic binary optimization problems using variational quantum eigensolvers (VQE).
  • Thumbnail Image
    A Multi-Agent Defense Methodology with Machine Learning against Cyberattacks on Distribution Systems
    Appiah-Kubi, Jennifer (Virginia Tech, 2022-08-17)
    The introduction of communication technology into the electric power grid has made the grid more reliable. Power system operators gain visibility over the power system and are able to resolve operational issues remotely via Supervisory Control And Data Acquisition (SCADA) technology. This reduces outage periods. Nonetheless, the remote-control capability has rendered the power grid vulnerable to cyberattacks. In December 2015, over 200,000 people in Ukraine became victims of the first publicly reported cyberattack on the power grid. Consequently, cyber-physical security research for the power system as a critical infrastructure is in critical need. Research on cybersecurity for power grids has produced a diverse literature; the multi-faceted nature of the grid makes it vulnerable to different types of cyberattacks, such as direct power grid, supply chain and ransom attacks. The attacks may also target different levels of grid operation, such as the transmission system, distribution system, microgrids, and generation. As these levels are characterized by varying operational constraints, the literature may be categorized not only according to the type of attack it targets, but also according to the level of power system operation under consideration. It is noteworthy that cybersecurity research for the transmission system dominates the literature, although the distribution system is noted to have a larger attack surface. For the distribution system, a notable attack type is the so-called direct switching attack, in which an attacker aims to disrupt power supply by compromising switching devices that connect equipment such as generators, and power grid lines. To maximize the damage, this attack tends to be coordinated as the attacker optimally selects the nodes and switches to attack. This decision-making process is often a bi- or tri-level optimization problem which models the interaction between the attacker and the power system defender. It is necessary to detect attacks and establish coordination/correlation among them. Determining coordination is a necessary step to predict the targets of an attack before attack completion, and aids in the mitigation strategy that ensues. While the literature has addressed the direct switching attack on the distribution system in different ways, there are also shortcomings. These include: (i) techniques to establish coordination among attacks are centralized, making them prone to single-point failures; (ii) techniques to establish coordination among attacks leverage only power system models, ignoring the influence of communication network vulnerabilities and load criticality in the decisions of the attacker; (iii) attacker-defender optimization models assume specific knowledge of the attacker resources and constraints by the defender, a strong unrealistic assumption that reduces their usability; (iv) and, mitigation strategies tend to be static and one-sided, being implemented only at the physical level, or at the communication network level. In light of this, this dissertation culminates in major contributions concerning real-time decentralized correlation of detected direct switching attacks and hybrid mitigation for electric power distribution systems. Concerning this, four novel contributions are presented: (i) a framework for decentralized correlation of attacks and mitigation; (ii) an attacker-defender optimization model that accounts for power system laws, load criticality, and cyber vulnerabilities in the decision-making process of the attacker; (iii) a real-time learning-based mechanism for determining correlation among detected attacks and predicting attack targets, and which does not assume knowledge of the attacker's resources and constraints by the power system defender; (iv) a hybrid mitigation strategy optimized in real-time based on information learned from detected attacks, and which combines both physical level and communication network level mitigation. Since the execution of intrusion detection systems and mechanisms such as the ones proposed in this dissertation may deter attackers from directly attacking the power grid, attackers may perform a supply chain cyberattack to yield the same results. Although, supply chain cyberattacks have been acknowledged as potentially far-reaching, and compliance directives put forward for this, the detection of supply chain cyberattacks is in a nascent stage. Consequently, this dissertation also proposes a novel method for detecting supply chain cyberattacks. To the best of the knowledge of the author, this work is the first preliminary work on supply chain cyberattack detection.
  • Thumbnail Image
    Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems
    Gu, Fangda; Yin, He; El Ghaoui, Laurent; Arcak, Murat; Seiler, Peter; Jin, Ming (2022)
    Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.
  • Thumbnail Image
    Sample Complexity of Incremental Policy Gradient Methods for Solving Multi-Task Reinforcement Learning
    Bai, Yitao (Virginia Tech, 2024-04-05)
    We consider a multi-task learning problem, where an agent is presented a number of N reinforcement learning tasks. To solve this problem, we are interested in studying the gradient approach, which iteratively updates an estimate of the optimal policy using the gradients of the value functions. The classic policy gradient method, however, may be expensive to implement in the multi-task settings as it requires access to the gradients of all the tasks at every iteration. To circumvent this issue, in this paper we propose to study an incremental policy gradient method, where the agent only uses the gradient of only one task at each iteration. Our main contribution is to provide theoretical results to characterize the performance of the proposed method. In particular, we show that incremental policy gradient methods converge to the optimal value of the multi-task reinforcement learning objectives at a sublinear rate O(1/√k), where k is the number of iterations. To illustrate its performance, we apply the proposed method to solve a simple multi-task variant of GridWorld problems, where an agent seeks to find an policy to navigate effectively in different environments.
  • Thumbnail Image
    TUNEOPT: An Evolutionary Reinforcement Learning HVAC Controller For Energy-Comfort Optimization Tuning
    Meimand, Mostafa; Khattar, Vanshaj; Yazdani, Zahra; Jazizadeh, Farrokh; Jin, Ming (ACM, 2023-11-15)
    HVAC systems account for the majority of energy consumption in buildings. Efficient control of HVAC systems can reduce energy consumption and enhance occupants’ comfort. In the existing literature, energy-comfort or cost-comfort co-optimization frameworks commonly involve manual tuning of the balancing coefficient between energy and comfort through parameter tuning by an expert. Nevertheless, achieving the optimal balance between energy usage and occupant comfort remains challenging. This limitation restricts the generalizability of different formulations across various scenarios or testing on different environments. In this paper, we propose an implicit evolutionary Reinforcement Learning (RL) approach to learn and adapt the trade-off parameter of an energy-comfort optimization formulation. We have developed a predictive comfortenergy co-optimization formulation for controlling the setpoint of a building. The RL agent utilizes a novel guidance-induced random search method to learn the energy-comfort trade-off coefficient and guide the optimization formulation. The reward function of the RL model is energy productivity (comfort over energy consumption). To evaluate the feasibility of our proposed approach, we conducted experiments on a real-world testbed - i.e., an apartment unit. Our feasibility study shows that the proposed approach can learn an optimal control parameter and reduce energy consumption by 24.3% while decreasing comfort by only 1% compared to the baseline.