ball off centre
—
loading…
commanded tilt
—
grip, weaker hand
—
physics rate
—
What you are looking at
Both arms hold one rigid object, so they are mechanically coupled: any disagreement between them is resolved as force through the grasp, and the grasp is what fails first. Once the jaws close, a single commanded tray pose determines both arms' targets — that is the coordination step, and it is why the whole system has a six-dimensional action space instead of fourteen joint angles.
The ball then makes the coupling matter. The tray's orientation is the
control input for a second, faster system riding on top of the first. A
PD law on the ball's position in the tray frame maps straight to a
commanded tilt:
accel = -12·offset − 4.5·velocity, then
pitch = accelx/g.
This page runs the classical controller, not the vision policy
Worth being straight about. The vision policy that produced the numbers below is trained on images from MuJoCo's own renderer; this page draws with three.js. Feeding it these pixels would be a different visual distribution — it would fail, and it would look like the policy failing rather than the setup being wrong. So the page runs the classical balancer, which reads the ball's state directly, and the learned results stay where they were actually measured:
| condition | classical | vision (2 × 84×112 images) | sight error |
|---|---|---|---|
| clean | 12/12 | 12/12 | 4.7 mm |
| camera noise | 12/12 | 12/12 | 5.7 mm |
| torque control at 40 % of rated | 12/12 | 12/12 | 4.6 mm |
| 0.5° backlash | 12/12 | 12/12 | 4.6 mm |
| all of them at once | 12/12 | 12/12 | 6.0 mm |
| no balancing (the floor) | 1/12 | ||
12 unseen randomized carries per row, randomized tray layouts, identical plans and seeds. The grasp is visual too: the handle posts are located from the same two cameras to 0.7 mm, and grasping from that estimate is indistinguishable from grasping from simulator state.
Everything — code, measurements, the bugs found along the way and the one conclusion that had to be corrected — is at github.com/ahmedsleem109/openarm-tray-carry. Every number is a simulation number; there is no real arm.