Current browse context:
eess.SY
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Systems and Control
Title: Control Policy Correction Framework for Reinforcement Learning-based Energy Arbitrage Strategies
(Submitted on 29 Apr 2024 (v1), last revised 30 Apr 2024 (this version, v2))
Abstract: A continuous rise in the penetration of renewable energy sources, along with the use of the single imbalance pricing, provides a new opportunity for balance responsible parties to reduce their cost through energy arbitrage in the imbalance settlement mechanism. Model-free reinforcement learning (RL) methods are an appropriate choice for solving the energy arbitrage problem due to their outstanding performance in solving complex stochastic sequential problems. However, RL is rarely deployed in real-world applications since its learned policy does not necessarily guarantee safety during the execution phase. In this paper, we propose a new RL-based control framework for batteries to obtain a safe energy arbitrage strategy in the imbalance settlement mechanism. In our proposed control framework, the agent initially aims to optimize the arbitrage revenue. Subsequently, in the post-processing step, we correct (constrain) the learned policy following a knowledge distillation process based on properties that follow human intuition. Our post-processing step is a generic method and is not restricted to the energy arbitrage domain. We use the Belgian imbalance price of 2023 to evaluate the performance of our proposed framework. Furthermore, we deploy our proposed control framework on a real battery to show its capability in the real world.
Submission history
From: Seyed Soroush Karimi Madahi [view email][v1] Mon, 29 Apr 2024 16:03:21 GMT (1961kb,D)
[v2] Tue, 30 Apr 2024 08:54:28 GMT (1961kb,D)
Link back to: arXiv, form interface, contact.