Towards a Unified Terminology for Benchmarking Bipedal Systems

Abstract

In the European project EUROBENCH, we are developing a framework for benchmarking the performances of bipedal systems: from humans to humanoids through wearable robots. Fair benchmarking requires defining and sharing clear and complete protocols so that bipedal systems can be studied and compared within similar and reproducible conditions. Even if the experimental methods and system comparisons are common scientific tasks, the description of the experimental protocols that are followed are rarely complete enough to allow it to be replicated. We list, in this article, the information required to properly define a protocol (e.g. experiment objectives, testbeds, type of collected and processed data, performance indicators used to score and compare experiments). Agreeing on a common terminology for benchmarking concepts will ease the evaluation of new technologies and promote communication between the different stakeholders involved in the development and use of bipedal systems

Publication
International Symposium on Wearable Robotics, WeRob2020, October 14-16

Supported by the H2020 EUROBENCH project, grant No. 779963.