Astronaut in space successfully controls ground-based rover

If man’s best friend is a dog, then in the future astronauts’ closest companions might well be rovers. A technique allowing astronauts in orbit to control rovers exploring planetary surfaces has been developed by a research team from ESA, the German Aerospace Center DLR and European academia and industry, culminating in an Earth-based rover session commanded from the International Space Station. A paper published in the Science Robotics journal this week details their results.

“This is the first time that an astronaut in space managed to control a robotic system on the ground in such an immersive, intuitive manner,” comments Aaron Pereira of DLR.

“Our 6 degree of freedom control interface incorporates force feedback so that the astronaut can experience just what the rover feels, even down to the weight and cohesion of the rocks it touches. What this does is help compensate for any limitations of bandwidth, poor lighting or signal delay to give a real sense of immersion—meaning the astronaut feels as though they are there at the scene.”

Let the robots take the strain

Robotics engineer Thomas Krueger, heading ESA’s Human Robot Interaction Laboratory, adds: “Robots can be given limited autonomy in known, structured environments, but for systems carrying out exploratory tasks such as sample collection in unknown, unstructured environments some kind of ‘human-in-the-loop’ oversight becomes essential. But direct control has not been feasible due to the inherent problem of signal delay—with transmission times constrained by the speed of light.

“So we have been working towards the concept of humans staying safely and comfortably in orbit around the Moon, Mars or other planetary bodies, but being close enough for direct oversight of rovers on the surface—combining the human strengths of flexibility and improvisation with a robust, dexterous robot on the spot to carry out their commands precisely.”

A team from ESA’s HRI Lab and DLR’s Robotics and Mechatronics Center collaborated on a series of progressively more complex tests, first on Earth then extending into orbit.

“In the end we needed to perform feasibility experiments from space because past research shows that weightlessness can degrade human performance during force and motion tasks,” adds Thomas. “This and other unique environmental factors meant that Earth-based simulations would not be sufficient.”

Their efforts culminated in the first part of the Analog-1 experiment in late 2019. Astronaut Luca Parmitano aboard the ISS operated the gripper-equipped ESA Interact rover in a mock lunar environment inside a hangar in Valkenburg, the Netherlands to survey rocks and collect samples. The two-hour space-to-ground test was a success, overcoming a two-way signal delay averaging more than 0.8 seconds and a data packet loss rate of 1% plus.

Solving the time factor

“Even though the ISS is in orbit just 400 km overhead, its signals are relayed to Earth via geostationary telecom satellites, then to Europe from Texas via a transatlantic cable,” explains Aaron.

“Our team at DLR had to design a control algorithm that could function on a stable basis despite this time lag. Because there is a delay in the force feedback received by the operator, they might continue to move the robot further even after it has hit a rock. This could lead to the robot going out of sync with its controller, potentially vibrating like crazy, perhaps even damaging itself.

“To prevent that happening, we use a concept called ‘passivity’—we look at the overall amount of energy an operator puts in, and on the remote side make sure the robot never gives out any more energy than that, and vice-versa. Like when you push a child on a swing, they are never going to go any higher than the first swing—and with friction and so on will swing gradually lower and lower.”

“So for instance, when the robot arm is moving and suddenly hits a rock it would take extra energy to move which the astronaut did not command, so we reduce the command energy at once to slow down the arm. Then, after the 850 microsecond delay, when the astronaut feels the rock they can then choose to add the extra energy to push it.

“This ‘Time Domain Passivity Approach for High Delay’ technique is very intuitive in practice and should be able to work well with higher delay times as well.”

Next steps on Mount Etna

Thomas concludes: “The main limitation of the work done so far is that our indoor analog lunar environment lacks realism. So this summer will see the second part of Analog-1 take place on the volcanic slopes of Mount Etna in Italy as part of a larger international robotic test campaign called ARCHES.

“Luca Parmitano will once again be controlling the Interact rover, this time in 1G from the ground.”