From Simulation to Reality: Deployment of Reinforcement Learning-Based Neural Network Controllers Trained with Modelica Models

Abstract

To address the limitations of traditional control methodsin complex systems, reinforcement learning (RL) combinedwith simulation models provides an efficient approach forcontroller development. In this work, we present a completetoolchain for developing and deploying RL-based neuralnetwork controllers using Modelica system models. Servingas a showcase, a real-world double-inverted pendulum isconstructed. The system was modeled in Modelica bycombining physics-based and data-driven modeling approachesfor efficient development. The hybrid model provides thetransition dynamics necessary for RL training. Couplingwith the RL environment is achieved through the FunctionalMock-up Interface (FMI) standard. Successful training andsim-to-real transfer are demonstrated on a single-pendulumsetup, validating the approach for extension to thedouble-inverted pendulum. This paper provides areproducible and extensible framework, well-suited foradvanced control tasks, and highlights the strengths ofModelica in combination with machine learning approaches.