Hot water systems are essential components in residential buildings, providing hot water for daily needs
and significantly impacting overall energy consumption. Controlling these systems is challenging due to their
use in different households with diverse needs and hot water production conditions, requiring a user-oriented
approach. Effective control must balance multiple conflicting objectives, such as minimizing energy costs
while ensuring user comfort. Additionally, hot water systems operate under uncertain conditions, influenced
by fluctuating demand, external temperature changes, and equipment degradation over time. Traditional
control methods often overlook these complexities, resulting in energy-intensive strategies and
overproduction of hot water to avoid discomfort.
This thesis explores optimal control strategies for managing hot water systems using Reinforcement
Learning (RL). The aim is to develop adaptive and generalizable control strategies for hot water systems,
addressing key challenges in controlling these systems and improving existing RL methods.
This research proposes an RL framework that replicates real-world conditions, allowing for the safe and
efficient development of control strategies through trial-and-error. Rather than developing a one-size-fits-all
approach, this work introduces new multi-objective RL methods to learn various control strategies adapted
to different households. In a second step, this work employs meta-reinforcement learning to enhance the
adaptability of these strategies, enabling them to generalize across diverse contexts and minimize the impact
of gaps between simulations and real-world conditions on performance. Finally, this work addresses
decision-making under uncertainties related to high variability in RL environments by developing risk-
sensitive control strategies that quantify and incorporate uncertainties in the learning process. Overall, this
thesis advances the application of RL for the control of dynamical systems, offering robust and adaptive
solutions for hot water systems across a range of household environments.
Keywords: Optimal control, Reinforcement learning, Multi-objective reinforcement learning, Meta-
learning, Uncertainty quantification, Hot water systems, Demand response

The jury will include:

- AMINI Massih-Reza, Professor, Université de Grenoble Alpes, Reviewer
- Chanel Caroline, Professor, ISAE-SUPAERO, Reviewer
- PAPADAKIS Panagiotis, Associate Professor, IMT Atlantique, Examiner
- PREUX Philippe, Professor, Université de Lille, Examiner
- BATTON-HUBERT Mireille, Professor, Mines Saint-Etienne, Thesis director
- LECOEUCHE Stéphane, Professor, IMT Mines Alès, Thesis director
- BOONAERT Jacques, Associate Professor, IMT Nord Europe, Co-supervisor
- VUILLAUME François, Engineer, FV-EI, Invited.