Generative Reward Dynamics in the Orienteering Problem: A Search-and-Rescue Application With Q-Learning
Advaith Jyothi Arun
Abstract
The orienteering problem (OP) and its many variants are standard models for route planning when rewards are uncertain. Almost all of these models share one assumption: the reward at a location is either fixed or changes on its own, independently of what the traveller does. This paper studies a different setting. Visiting a vertex does not reveal a colour that was already waiting there; instead, it triggers the random colouring of other, previously uncoloured vertices. Reward generation is therefore driven by the agent's own movement, which separates this model from earlier dynamic-reward work. We formalise the idea as a ternary reward system on a graph, where each vertex is uncoloured, blue (a survivor, +1), or red (a hazard, −1), and we instantiate it as a search-and-rescue game in which a rescue vehicle runs on a limited fuel budget and a limited carrying capacity, collecting survivors while avoiding hazards. The model is built up from the classical OP, the dynamic OP with random rewards, and the robot-routing problem for aggregate stochastic rewards, and we show that the offline version reduces to a budget-constrained, return-to-depot OP and is therefore NP-hard. We then compare two policies over 800 missions on randomly generated maps: a memoryless random walk and a tabular Q-learning agent. Q-learning improves the mean mission score nearly sevenfold over the random walk (about 6.036 against 0.863), rescuing more survivors and running into far fewer hazards. We close by discussing the model's limitations and directions for future work.
