Green Urban Mobility with Autonomous Electric Ferries: Studies of Simulated Maritime Collisions using Adaptive Stress Testing

Abstract

With 90% of the world's goods transported by sea vessels, it is crucial to investigate their safety. This is increasingly important as autonomy is being introduced into sea vessels, which transport goods and people. To study the safety of an autonomous ferry's collision avoidance system, we consider the Adaptive Stress Testing (AST) method in this work. AST uses machine learning, specifically reinforcement learning, along with a simulation of a system under test---in our case, an autonomous and electric ferry---and its environment. Whether that simulation is fully or partially observable has implications for the integration into existing engineering workflows. The reason is that the fully observable simulation induces a more complex interface than the partially observable simulation, meaning that the engineers designing and implementing AST need to acquire and comprehend more potentially complex domain knowledge. This paper presents maritime adaptive stress testing (MAST) methods, using the world's first autonomous, electric ferry used to transport people as a case study. Using MAST in multiple scenarios, we demonstrate that AST can be productively utilized in the maritime domain. The demonstration scenarios stress test a maritime collision avoidance system known as Single Path Velocity Planner (SP-VP). Additionally, we consider how MAST can be implemented to test using both fully observable (gray box) and partially observable (black box) simulators. Consequently, we introduce the Gray-Box MAST (G-MAST) and Black-Box MAST (B-MAST) architectures, respectively. In simulation experiments, both architectures successfully identify an almost equal number of failure events. We discuss lessons learned about MAST including the experiences with both the Gray-Box and Black-Box approaches.