1: \begin{abstract}

2:     Federated learning (FL) enables distributed cross-client training while preserving the privacy of clients' training data, eliminating the need to disclose it to a central authority or other participants. However, the distributed nature of FL exposes it to potential attacks from adversarial (Byzantine) clients who may modify their local updates to suit their own interests. Therefore, it becomes essential to develop robust FL algorithms that can defend against Byzantine clients while maintaining model convergence and performance. Despite various studies on Byzantine-resilient FL from both attack and defense aspects, there is a lack of unified benchmark suites to facilitate and accelerate the fair comparison and evaluation of different techniques. 

3: 

4:     This paper investigates existing techniques in Byzantine-resilient FL and introduces an open-source benchmark suite\footnote{Upon acceptance of the paper, the latest version of our suite will undergo artefact evaluation and then be made available as an open-source resource.} for convenient and fair performance comparisons. Our investigation begins with a systematic study of Byzantine attack and defense strategies. Subsequently, we present \ours, a scalable, extensible, and easily configurable benchmark suite that aims at supporting researchers and developers in efficiently implementing and validating novel strategies against baseline algorithms in the domain of Byzantine-resilient FL. The design of \ours incorporates key characteristics derived from our systematic study, encompassing the attacker's capabilities and knowledge, defense strategy categories, and factors that influence robustness. 

5:     Blades contains built-in implementations of representative attack and defense strategies and offers user-friendly interfaces for seamless integration of new ideas.

6:     % We maintain the source code and documents at \url{https://github.com/lishenghui/blades}.

7: \end{abstract}

8: