Current browse context:
cs.RO
Change to browse by:
References & Citations
Computer Science > Robotics
Title: SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution
(Submitted on 18 Dec 2023 (v1), last revised 28 Mar 2024 (this version, v3))
Abstract: Diffusion models have demonstrated strong potential for robotic trajectory planning. However, generating coherent trajectories from high-level instructions remains challenging, especially for long-range composition tasks requiring multiple sequential skills. We propose SkillDiffuser, an end-to-end hierarchical planning framework integrating interpretable skill learning with conditional diffusion planning to address this problem. At the higher level, the skill abstraction module learns discrete, human-understandable skill representations from visual observations and language instructions. These learned skill embeddings are then used to condition the diffusion model to generate customized latent trajectories aligned with the skills. This allows generating diverse state trajectories that adhere to the learnable skills. By integrating skill learning with conditional trajectory generation, SkillDiffuser produces coherent behavior following abstract instructions across diverse tasks. Experiments on multi-task robotic manipulation benchmarks like Meta-World and LOReL demonstrate state-of-the-art performance and human-interpretable skill representations from SkillDiffuser. More visualization results and information could be found on our website.
Submission history
From: Zhixuan Liang [view email][v1] Mon, 18 Dec 2023 18:16:52 GMT (3239kb,D)
[v2] Wed, 13 Mar 2024 16:29:50 GMT (3893kb,D)
[v3] Thu, 28 Mar 2024 16:49:40 GMT (3724kb,D)
Link back to: arXiv, form interface, contact.