Moving beyond Competitions: Extending D-Cube to Seamlessly Benchmark Low-Power Wireless Systems

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Performance comparisons of low-power wireless systems are often not substantiated by accurate and realistic evaluations, which raises the need of a proper benchmark. In a first attempt towards a rigorous comparison of protocol performance under the exact same settings, we have developed in 2016 a prototype benchmarking infrastructure called D-Cube, and used it to run the first of a series of competitions aiming to quantitatively assess the performance of low-power wireless protocols in specific scenarios. Given the success of the competition among both academia and industry, we have significantly extended the benchmarking infrastructure in the following two editions: D-Cube now also supports, among others, remote experimentation, multiple traffic patterns and loads, a custom description of how to derive performance metrics, and is further able to control the network density as well as the harshness of the RF environment. In this paper we perform a critical analysis of the current capabilities of D-Cube and argue that its main limiting factor is that the traffic patterns and node identities are manually embedded in the source code by developers and cannot be changed automatically. We show that we can overcome this limitation by utilizing a well-known data structure and by having developers describe its memory address using a configuration file that is passed to the benchmarking infrastructure. Following this concept, we extend D-Cube with the ability of building and applying patches to binary files and show that this allows not only to automatically change traffic patterns and node identities, but to also change user-defined protocol parameters. We believe that this extension is one of the last missing stepping stones to make D-Cube a full-fledged benchmarking infrastructure for low-power wireless systems.

Original languageEnglish
Title of host publicationProceedings - 2018 1st Workshop on Benchmarking Cyber-Physical Networks and Systems, CPSBench 2018
PublisherInstitute of Electrical and Electronics Engineers
Pages30-35
Number of pages6
ISBN (Print)9781538667422
DOIs
Publication statusPublished - 7 Aug 2018
Event1st Workshop on Benchmarking Cyber-Physical Networks and Systems, CPSBench 2018 - Porto, Portugal
Duration: 10 Apr 2018 → …

Conference

Conference1st Workshop on Benchmarking Cyber-Physical Networks and Systems, CPSBench 2018
CountryPortugal
CityPorto
Period10/04/18 → …

Keywords

  • benchmarking
  • competition
  • dependability
  • IoT
  • low power wireless
  • measurement
  • performance
  • testbeds

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Safety, Risk, Reliability and Quality

Fields of Expertise

  • Information, Communication & Computing

Fingerprint Dive into the research topics of 'Moving beyond Competitions: Extending D-Cube to Seamlessly Benchmark Low-Power Wireless Systems'. Together they form a unique fingerprint.

Cite this