Abstract
Generalising vision-based manipulation policies to novel environments remains a challenging area with
limited exploration. Current practices involve collecting data in one location, training imitation learning
or reinforcement learning policies with this data, and deploying the policy in the same location. However,
this approach lacks scalability as it necessitates data collection in multiple locations for each task. This
paper proposes a novel approach where data is collected in a location predominantly featuring green screens.
We introduce Green-screen Augmentation (GreenAug), employing a chroma key algorithm to overlay background
textures onto a green screen. Through extensive real-world empirical studies with over 850 training
demonstrations and 8.2k evaluation episodes, we demonstrate that GreenAug surpasses no augmentation,
standard computer vision augmentation, and prior generative augmentation methods in performance. While no
algorithmic novelties are claimed, our paper advocates for a fundamental shift in data collection practices.
We propose that real-world demonstrations in future research should utilise green screens, followed by the
application of GreenAug. We believe GreenAug unlocks policy generalisation to visually distinct novel
locations, addressing the current scene generalisation limitations in robot learning.