Expert imitation for sequential decision making

Imitation learning, where an agent learns decision-making by mimicking expert behavior, has been widely applied in robotics and autonomous driving. This study introduces an innovative approach using DAgger, an imitation learning algorithm, to iteratively train policies for stochastic sequential decision problems (SDPs). These problems are particularly challenging when expert input is costly or unavailable. First, we define a taxonomy of experts that can be used for imitation in SDPs. We then focus on designing an effective expert within the DAgger framework, leveraging deterministic solutions derived from contextual scenarios at each decision point. A predictive model is subsequently trained to replicate the expert’s behavior, enabling real-time decision-making. However, imperfect experts may provide faulty demonstrations, slowing down learning. To mitigate this, we propose a strategy incorporating multiple experts within the DAgger framework. To demonstrate the methodology’s applicability, we address a dynamic employee notification timing problem for scheduling casual personnel in on-call shifts.