Abstract
Most Reinforcement Learning (RL) environments are created by adaptingexisting physics simulators or video games. However, they usually lack theflexibility required for analyzing specific characteristics of RL methods oftenrelevant to research. This paper presents Craftium, a novel framework forexploring and creating rich 3D visual RL environments that builds upon theMinetest game engine and the popular Gymnasium API. Minetest is built to beextended and can be used to easily create voxel-based 3D environments (oftensimilar to Minecraft), while Gymnasium offers a simple and common interface forRL research. Craftium provides a platform that allows practitioners to createfully customized environments to suit their specific research requirements,ranging from simple visual tasks to infinite and procedurally generated worlds.We also provide five ready-to-use environments for benchmarking and as examplesof how to develop new ones. The code and documentation are available athttps://github.com/mikelma/craftium/.