Abstract
Reinforcement learning (RL) algorithms, due to their reliance on externalsystems to learn from, require digital environments (e.g., simulators) withvery simple interfaces, which in turn constrain significantly theimplementation of such environments. In particular, these environments areimplemented either as separate processes or as state machines, leading tosynchronization and communication overheads in the first case, and tounstructured programming in the second. We propose a new domain-specific, co-routine-based, compiled language, calledRulebook, designed to automatically generate the state machine required tointeract with machine learning (ML) algorithms and similar applications, withno performance overhead. Rulebook allows users to express programs withoutneeding to be aware of the specific interface required by the ML components. Bydecoupling the execution model of the program from the syntactical encoding ofthe program, and thus without the need for manual state management, Rulebookallows to create larger and more sophisticated environments at a lowerdevelopment cost.