How it works
A pick and pour station with two SO-101 arms, a leader and a follower, and two cameras. We recorded demos by moving the leader arm by hand and trained ACT policies on them.
We also built copies of the station in MuJoCo and Isaac Sim, and made synthetic demos with Isaac Lab Mimic on a rented GPU.
On top sits a slow vision model that makes the decisions while a fast policy moves the arm. In one test an OpenAI model picks a can and puts it back. There is a bartender dashboard too.
Stack
SO-101 arms / LeRobot / LeLab / ACT / SmolVLA / MuJoCo / Isaac Sim / Isaac Lab Mimic / OpenAI / Python
