High performance locomotion

The speed of walking robots is a critical factor in many applications, especially in autonomous robot soccer games. Our aim is to increase the walking speed of our robots through automated hardware-in-the-loop optimization.

Many different approaches have been investigated for improving the walking speed of bipedal and quadrupedal robots. For example, numerical optimal control techniques enable the computation of stable and fast walking motions based on three dimensional computational models of the legged robot dynamics. However, all model-based optimization approaches have in common that their outcome critically depends on the quality and accuracy of the robot model. The derivation of highly accurate enough robot models to achieve the best possible walking speed may require too many efforts considering, e.g., the effects of gear backlash, elasticity and temperature dependent joint friction or of different ground properties. An alternative approach is to start with a reasonable, initial walking motion and then to use online hardware-in-the-loop optimization of the physical robot. For example, reinforcement learning techniques for four-legged and biped robots have successfully been used to improve fast and stable walking motions, where a (low dimensional) parametrization of the walking motion is utilized to optimize the gait by evaluating the merit function performing walking experiments with the real robot.

In the following our current work on quadrupedal (videos) and bipedal (videos) walking optimization is described briefly.

Walking optimization of four-legged Sony Aibo robots

Optimization for our robots of type Sony Aibo was performed in two scenarios, (1) straight foward walking without ball, (2) turning around (it's vertical axis) without losing the ball, whereby the ball is placed in between the front legs of the robot, and it has to stay there during the turning motion. These two cases often arise during a robot scoccer game, the first when the robot has to reach the ball, the second one when it is aligning for passing the ball to another robot, or when it tries to shoot a goal. The walking trajectories of the four legs are determined in space and time by a 31 dimensional parameter vector. This low dimensional parametrization is achieved by approximating the trajectory by polygons and by taking advantage of symmetry and redundancies of a four legged walking robot. The walking optimization is performed on the standard RoboCup 4-legged league field, where the walking or turning speed of the robot is measured by a ceiling camera. The position and orientation of the robot is detected using circular markers attached to the robot's back. The speed of the robot is then approximated from two consecutive measurements over a fixed time interval. The average speed measured over a fixed number of runs is returned to the optimizer as the objective function value. When optimizing turning while holding the ball, a distance sensor in the chest of the robot is applied in order to determine whether the ball was lost while walking. For such an unsuccessful run, an evaluation value of zero is returned to the optimizer. Therefore walking motions are penalized where the ball is not held securely.

This way a hardware-in-the-loop layout is applied, where the robot control is coupled with the optimization tool, in our case APPSPACK (ver. 4.0.2). This method is originally designed for deterministic nonlinear optimization problems, but for the here considered problem with averaged walking speed respectively turning speed the chosen optimization method performed well. Besides the restricting factor in this application is the number of walking experiments, so that no implemented stopping criteria of APPSPACK is used, which would also be disturbed by the underlying stochastic character of this application.

The starting parameter set for the forward walking speed optimization is obtained by an optimization experiment with an evolutionary algorithm, for the turning speed optimization we start with a hand tuned initial parameter set. Beside of bounds for the parameter space representing reachable leg positions, we are using the default parameters of APPSPACK.

For walking forward the measured speed of the initial parameter set was 40 cm/s. The resulting parameters obtained by optimization using APPSPACK was measured at an average speed of 43 cm/s. Thus yielding an improvement in the walking speed of about 7.5%. This result was achieved after evaluating 83 parameter sets in about half an hour optimization time. When optimizing the turning motion of the robot, the turning speed could be increased about 50% from 120 deg/s to 180 deg/s without losing the ball while turning. Finding this result required 206 parameter set evaluations which took about 45 minutes.

Videos: Forward walking, turning

As quality or objective function value of a walking motion we measure the distance the robot covers, when it starts walking with a small step length and increases it linearly during the experiment until the robot falls or reaches a final step length.

During the numerical online optimization the real-valued parameters influencing the main characteristics of the walking behavior of the robot are varied to maximize the defined objective function. The robot is included as hardware-in-the-loop for the walking experiment to evaluate the objective function. In this context a non deterministic black-box optimization problem arises, where besides of a noise function value no further information, especially no objective gradients, is provided. By this definition of the objective function, it is not necessary to formulate additional constraints for maintaining walking stability and incorporate them explicitly into the definition of the optimization problem. The only constraints to be considered are the constant lower and upper bounds on the optimization parameters.

Video: Initial walk

Video: Optimized walk

Video: Optimized walk,
improved robot