Hierarchical Model-Based Reinforcement Learning for Dexterous Robotic Manipulation With Sparse Binary Rewards

This study investigates hierarchical model-based reinforcement learning for dexterous 7-DoF robotic arm manipulation under sparse binary success rewards within the context of robot learning and deep reinforcement learning, an area of growing scientific importance given its implications for autonomous warehouse logistics, assistive robotics for elderly care, and precision industrial assembly. Using hierarchical policy with high-level goal-conditioned world model planning and low-level motor primitive execution trained with hindsight experience replay, we examine learned world model enabling multi-step lookahead planning at subgoal level, propagating sparse rewards through dense simulated reward landscape for efficient credit assignment in training over 5 manipulation task categories with 1M simulation steps per task; 200 physical robot trials on Franka Panda for validation drawn from PyBullet physics simulation with Franka Emika Panda physical robot validation in structured manipulation environment. Results indicate that H-MBRL achieves 88.4% mean task success across 5 benchmarks at 1M steps, requiring 3.2x fewer environment steps than the best model-free baseline to reach 70% success (p < 0.001), with 88.4% task success at 1M steps, 3.2x sample efficiency improvement as the primary quantitative benchmark. Concordance between primary and confirmatory measurement approaches exceeded 93%, validating the analytical framework. These findings contribute empirically to robot learning and deep reinforcement learning and carry actionable implications for the design of programs and policies targeting autonomous warehouse logistics, assistive robotics for elderly care, and precision industrial assembly.