Teaching a Robot Hand to Walk

Although it wasn’t apparently designed with this in mind, it seems particularly fortuitous that this walking robotic hand was released in time for Halloween. Designed by researchers from ETH Zurich, …. Designed by researchers from ETH Zurich, the slightly unsettling disembodied hand can use its fingers as legs to traverse terrain, push small objects around, and operate a keyboard.

The researchers started from commercially-available robot hand, equipped it with a battery and Raspberry Pi Zero 2 W, and developed neural net-based software to control it. The hand has twenty joints, four per finger, and the neural net iteratively outputs the next joint state, based on previous movements, the state of the hand, and the hand’s current goal. To train the net, the researchers built a simulated model, then used this for reinforcement learning; this yielded a faster walking speed than an adapted quadrupedal motion model did.

The hand was trained to move in a straight line, turn, recover from a fall, and press the keys of a keyboard (since it doesn’t have a camera, though, it can’t operate a keyboard by itself). The fall recovery is impressive to watch: in 21 out of 25 tests, it was able to right itself within twenty seconds. Due to the hand’s geometry, it drifts to the right while walking, so a constant correction needed to be applied.