mani-wav.github.io - ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data

Example domain paragraphs

Stanford University &nbsp&nbsp Columbia University &nbsp&nbsp Toyota Research Institute

1 Stanford University &nbsp&nbsp&nbsp&nbsp 2 Columbia University &nbsp&nbsp&nbsp&nbsp 3 Toyota Research Institute

Audio signals provide rich information for the robot interaction and object properties through contact. These information can surprisingly ease the learning of contact-rich robot manipulation skills, especially when the visual information alone is ambiguous or incomplete. However, the usage of audio data in robot manipulation has been constrained to teleoperated demonstrations collected by either attaching a microphone to the robot or object, which significantly limits its usage in robot learning pipelines.

Links to mani-wav.github.io (1)