In the European project EUROBENCH, we are developing a framework for benchmarking the performances of bipedal systems: from humans to humanoids through wearable robots. Fair benchmarking requires defining and sharing clear and complete protocols so that bipedal systems can be studied and compared within similar and reproducible conditions. Even if the experimental methods and system comparisons are common scientific tasks, the description of the experimental protocols that are followed are rarely complete enough to allow it to be replicated. We list, in this article, the information required to properly define a protocol (e.g. experiment objectives, testbeds, type of collected and processed data, performance indicators used to score and compare experiments). Agreeing on a common terminology for benchmarking concepts will ease the evaluation of new technologies and promote communication between the different stakeholders involved in the development and use of bipedal systems
Supported by the H2020 EUROBENCH project, grant No. 779963.