Energy management of a microgrid considering nonlinear losses in batteries through Deep Reinforcement Learning

D. Domínguez-Barbero, J. García-González, M.A. Sanz-Bobi, A. García-Cerrada

Applied Energy Vol. 368, pp. 123435-1 - 123435-12

Summary:

The massive deployment of microgrids could play a significant role in achieving decarbonization of the electric sector amid the ongoing energy transition. The effective operation of these microgrids requires an Energy Management System (EMS), which establishes control set-points for all dispatchable components. EMSs can be formulated as classical optimization problems or as Partially-Observable Markov Decision Processes (POMDPs). Recently, Deep Reinforcement Learning (DRL) algorithms have been employed to solve the latter, gaining popularity in recent years. Since DRL methods promise to deal effectively with nonlinear dynamics, this paper examines the Twin-Delayed Deep Deterministic Policy Gradient (TD3) performance – a state-of-the-art method in DRL – for the EMS of a microgrid that includes nonlinear battery losses. Furthermore, the classical EMS-microgrid interaction is improved by refining the behavior of the underlying control system to obtain reliable results. The performance of this novel approach has been tested on two distinct microgrids – a residential one and a larger-scale grid – with a satisfactory outcome beyond reducing operational costs. Findings demonstrate the intrinsic potential of DRL-based algorithms for enhancing energy management and driving more efficient power systems.

Spanish layman's summary:

Modelar adecuadamente las baterías debería ayudar a operar eficientemente el sistema eléctrico. Las técnicas de aprendizaje por refuerzo profundo (DRL) operan sin dificultad con modelos no lineales. Este trabajo estudia el impacto del DRL en una micro-red aislada operando con un modelo no lineal de baterías.

English layman's summary:

Modelling batteries properly should help operate the electrical system efficiently. Deep reinforcement learning (DRL) techniques handle nonlinear models with ease. This work studies the impact of DRL on an isolated microgrid operating with a nonlinear battery model.

Keywords: Deep Reinforcement Learning; Energy management system; Energy savings; Isolated microgrid; Nonlinear battery model

JCR Impact Factor and WoS quartile: 11,200 - Q1 (2022)

DOI reference: https://doi.org/10.1016/j.apenergy.2024.123435

Published on paper: August 2024.

Published on-line: May 2024.

Citation:
D. Domínguez-Barbero, J. García-González, M.A. Sanz-Bobi, A. García-Cerrada, Energy management of a microgrid considering nonlinear losses in batteries through Deep Reinforcement Learning. Applied Energy. Vol. 368, pp. 123435-1 - 123435-12, August 2024. [Online: May 2024]