In what marks significant progress for robotic manipulation of real-world objects, a Jenga-playing machine can learn the complex physics involved in withdrawing wooden blocks from a tower through physical trial and error.
This differentiates it from robots that have mastered purely cognitive games such as chess and Go through visual cues.
“Playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing and aligning pieces,” said Prof Alberto Rodriguez from the department of mechanical engineering at Massachusetts Institute of Technology.
Combining interactive perception and manipulation – whereby the robot would touch the tower to learn how and when to move blocks – is extremely difficult to simulate and therefore the robot has to learn in the real world, he added. So the researchers placed a two-pronged industrial robot arm with a force sensor in its wrist by the Jenga tower and allowed it to explore rather than using traditional machine-learning techniques that could require data from tens of thousands of block-extraction attempts in order to capture every possible scenario.
The robot grouped the outcomes of approximately 300 attempts as it discovered that some blocks were harder to budge than others. “The robot builds clusters and then learns models for each of these clusters, instead of learning a model that captures absolutely everything that could happen,” said the paper’s lead author, MIT graduate student Nima Fazeli.
This enabled the robot to develop a simple model to predict a block’s behaviour on the basis of its visual and tactile measurements as it gained an appreciation of the dynamics behind Jenga.
“There are many tasks that we do with our hands where the feeling of doing it the right way comes in the language of forces and tactile cues,” Rodriguez said. “For tasks like these, a similar approach to ours could figure it out.”
Still, the bot will have to improve if it is to conquer a human player, but it is not far off. Miquel Oller, a member of the team, said: “We saw how many blocks a human was able to extract before the tower fell and the difference was not that much.”
