What can a robot really do? 6 things to look out for in videos

It is just die Time for humanoid robots. At least that’s how it seems to Ken Goldberg, a busy roboticist who heads research in this field at the University of California, Berkeley, and has founded several robotics companies.

A lot of money is currently flowing into humanoid robot startups, including Figure AI, which raised $675 million earlier this year. Agility Robotics, on the other hand, has left the pilot phase behind and says it has put its first fleet of humanoid robots into operation at a Spanx factory in Georgia.

Robot videos as a marketing tool

But what really drives home the feeling of a humanoid robot heyday are the videos. Seemingly every month there’s a new moody, futuristic video in which a humanoid stares intensely (or exasperatedly) into the camera, jumps around, or sorts things into piles. Sometimes they even talk.

Such videos are all the rage in robotics right now. As Goldberg says, you can’t just fire up a humanoid robot at home and play around with it, as you can with the latest version of ChatGPT. For anyone looking to ride the AI ​​wave or demonstrate their progress – like a startup or an academic seeking lab funding – a good video featuring humanoids is the best marketing tool available. “The images, the visuals and the videos – they played a big role,” he says.

If you look at a few dozen from this year, you often find yourself feeling agitated, shocked, or even bored. What exactly should you look out for? Goldberg has some tips for classifying things.

1. Remote controlled or not?

First, a few basics. The most important question is whether a robot is remotely controlled by a human from off-screen and does not perform the tasks independently. Unfortunately, this cannot always be determined with certainty unless the company discloses it in the video – which is not always the case. In a video published by Tesla at the beginning of the year, however, you can see that a control glove appears every now and then on the right edge of the screen, which carries out the movements for the robot.

2. Selecting the video

The second problem is selection bias. How many shots were necessary to get the perfect shot? If a humanoid shows an impressive ability to sort objects but needs 200 attempts to successfully complete the task, this helps to realistically assess its hit rate.

3. Is the video recorded in time lapse?

Finally, was the video sped up? Often, that’s perfectly fine if it skips over things that don’t say much about the robot (“I don’t want to watch the paint dry,” Goldberg says). However, if the video is sped up to intentionally hide something or make the robot seem more effective than it actually is, that’s something to keep in mind when evaluating capability. All of these editing decisions should ideally be disclosed by the robot company or lab.

4. Pay attention to the robot hands

In videos of humanoid robots, the robot’s hands are shown off by the fingers gently clenching into a fist. A robotic hand with so many usable joints is indeed more complex than the grippers of industrial robots, Goldberg says, but these humanoid hands may not be able to do what the videos sometimes suggest.

For example, humanoid robots are often shown holding a box while walking. The footage might give the impression that they are using their hands like humans, sliding their fingers under the box and lifting it. But often, Goldberg says, the robots are just pushing the box horizontally, with the force coming from the shoulder. It still works, but not in the way you’d imagine. In most videos, the hands aren’t visible at all, which isn’t surprising, since the dexterity of the hands requires an enormously complicated technique.

5. What does the environment look like?

The latest humanoid videos show that robots are getting better at walking and even running. “A robot that can run faster than a human is probably on the verge of a breakthrough,” says Goldberg.

Still, it’s important to pay attention to what the robot’s environment looks like in the video. Is there clutter or dust on the floor? Are people getting in its way? Are there stairs, equipment, or slippery surfaces in its path? Chances are, there aren’t. The robots generally demonstrate their – admittedly impressive – feats in pristine environments, but not in warehouses, factories, and other places where they will supposedly work alongside people.

We report on the progress robotics has actually made thanks to AI in issue 5/2024 of MIT Technology Review. You can order TR 5/2024 as a print and PDF edition here.

6. Tricks for a robot’s payload

Humanoids are sometimes not as strong as the videos of their physical feats suggest. Many of them can’t even hold a hammer at arm’s length. They can carry more if they keep the weight close to their core, but their carrying capacity fluctuates dramatically when they extend their arms. Watch how robots move boxes from one conveyor belt to another, because those boxes could be empty.

With all the hype surrounding humanoid robots, there are countless other questions to be answered, not least the question of how much these machines will ultimately cost. But perhaps this classification at least offers a better perspective on the robots that are becoming increasingly important.

The article was written by James O’Donnell. He is an editor at the US edition of MIT Technology Review. O’Donnell regularly writes about hardware and AI topics.

Examples of robots in use