Full day workshop at IEEE Humanoids
jointly organized by the European projects H2R, KoroiBot, and WalkMan

Place and date

Nov 18, 2014,  9:00 – 18:00
Humanoids Conference 2014
Madrid, Spain


  • Diego Torricelli, CSIC, Spain,
  • Katja Mombaur, University of Heidelberg,,
  • Alexander Schubert, University of Heidelberg,
  • Jose L. Pons, CSIC, Spain,
  • Nikos Tsakarakis, IIT, Italy
  • Zhibin Li, IIT, Italy

 Workshop objectives

Achieving stable, efficient and versatile locomotion is one of the main goals of humanoid robots. In this context, the need for reliable benchmarks of bipedal locomotion is becoming more and more relevant, in order to assess the robotic behavior on a quantitative basis and to compare it with the human counterpart.
Such benchmarks or performance indicators are important in order to judge when a robot motion is “human-like” which is an often stated but never strictly defined goal. They are also important in order to compare the abilities of different robots to each other or to measure progress of an individual robot due to an implementation of new control methods or an exchange or parts. Despite this importance, no consensus has been achieved yet about the set of benchmarks that should be adopted.
This workshop aims to involve the international community interested in bipedal walking in a thorough discussion about the key features of human-like locomotion, with a specific focus on the assessment methods and protocols that can be easily applied across different laboratory settings. The aim of this workshop is to really WORK together and to come up, at the end of the day, with a preliminary, but specific proposal for benchmarks for humanoid walking.
In order to achieve this objective, the organization of this workshop will deviate from the usual structure of IEEE conference workshops based on plenary talks and discussions. After several very short challenge talks, the workshop will essentially be structured in small discussion groups, in order to promote real debate and reasoning, which possibly result in a preliminary proposal of benchmarks to be proposed to the international forum. The discussion groups will be focusing on different aspects related to functional performance and human-likeness, such as stability, efficiency, dynamic similarity, passive dynamics, etc. The discussion groups will then present their results to all workshop participants.
In order to be able to be really productive, this workshop will be well prepared via the newly established benchmarking mailing list and this webpage; and abstracts corresponding to the challenge talks will already be made available to the prospective audience in advance.
We strongly encourage the participation of all those researchers interested in sharing their ideas and willing to find common solutions.

 Intended Audience:

This workshop aims at gathering audience (researchers, students and industry) that is interested in the challenges of benchmarking bipedal locomotion, especially from the fields of:

  • humanoid and other bipedal/legged robots
  • lower limb exoskeletons
  • cognitive sciences
  • biomechanics
  • medicine

We would like to emphasize that due to the particular organization structure of the workshop with small discussion groups, participants who wish to actively contribute to the workshop goals are encouraged to participate the whole day and not to plan to perform workshop hopping.


Topics of the individual discussion groups will be chosen by the attendees and will include:

  • Identification of requirements and associated performance metrics for bipedal locomotion and balance
  • development of a common taxonomy for qualitative and quantitative analysis of bipedal locomotion
  • definition of standard test methods and procedures to  validate bipedal locomotion skills and performance benchmarking of stability during gait and posture
  • benchmarking of flexibility/versatility
  • benchmarking  of  energy consumption and efficiency
  • benchmarking of cognitive abilities such as prediction and anticipation of disturbances

Final Program

09:00 – 09:10              Introduction, goals of the workshop (Organizers)
09:10 –  10:30             Challenge talks (part 1)
10:30 – 11:00             Coffee  break & poster session
11:00 –  11:30             Challenge talks (part 2)
11:30 – 12:30             Split of participants in discussion groups, 1st round of discussions
12:30 – 14:00             Lunch break
14:00 – 14:15             Short statements of all discussions groups about 1st round results
14:15 – 15:45             2nd round of discussions
15:45 – 16:30             Coffee break & poster session
16:30 – 17:00             Presentation of results of different discussion groups
17:00 – 18:00             Final discussion

List of challenge talk speakers and topics

Part 1

  • Franco Mastroddi (EU) –
    Title: “Benchmarking and performance evaluation in robotics: a European contribution”
  • Katja Mombaur (University of Heidelberg, Germany, KoroiBot project) –
    Title: “Benchmarking of humanoid locomotion performance in Koroibot project”
  • Olivier Stasse (LAAS-CNRS, France, KoroiBot project) –
    Title: “Measuring key performance indicators for HRP-2 in different walking situations”
  • Jose Gonzalez (CSIC, Spain, H2R project) –
    Title: “Benchmarking human-likeness of bipedal locomotion”
  • Jan Veneman (TECNALIA, Spain, BALANCE project) –
    Title: “Web-based survey on benchmarking: preliminary results”

Part 2

  • Nikos Tsagarakis (IIT, Italy, Walkman project) –
    Title: “Development of a High Performance, Compliant and Efficient Humanoid: Design specifications and Validation tasks in close Interaction”
  • Diego Torricelli (CSIC, Spain, H2R project) –
    Title: “A proposal of benchmarking scheme for bipedal walking and posture”

Poster/interactive presentations

There will be interactive presentations about several topics related to benchmarking and about the European projects associated with the workshop in the two coffeee breaks.
A list of all interactive presentations will be available here soon.

Supporting Projects and TCs:

The workshop is organized by the European projects H2R, KoroiBot and WALKMAN.
It is supported by the recently created “list on benchmarking bipedal locomotion”.
This workshop will be supported by the IEEE RAS Technical committee on Model-based optimization for Robotics.