Safe and Efficient Control of a Brayton Cycle Heat Pump Using Reinforcement Learning

Abstract

Decarbonizing industrial process heating will increasinglydepend on high-temperature heat pumps. In particular,Brayton cycle heat pumps, which can reach temperaturesabove 250 °C, are viewed as a promising technology.However, ensuring safe operation and optimal controlremains challenging. This study presents an experimentallyvalidated dynamic model of a Brayton cycle heat pump, asystem with multiple control inputs for regulating itsthermal output. Using this model as a training environment,several control concepts integrating Reinforcement Learning(RL) and traditional PI controllers were implemented toachieve desired heat supply at target temperatures. Domainrandomization was employed to improve the controllerrobustness against model uncertainties in preparation fordeployment on the physical system. The results demonstratethat RL controllers can not only achieve the desiredset-point temperature under varying loads while maintainingrequired safety margins, but also discovered a novel, moreenergy-efficient operational strategy.