Fair Federated Learning

Content of this post is based on chapter six of this publication.

Federated learning raises several opportunities for fairness research, some of which extend prior research directions in the non-federated setting, and others that are unique to federated learning.

Bias in Training Data

One driver of unfairness in machine-learning models is bias in the training data, including cognitive, sampling, reporting, and confirmation bias. One common anti-pattern is that minority or marginalized social groups are under-represented in the training data, and thus the learner weights these groups less during training, leading to inferior quality predictions for members of these groups. Just as the data access process used in federated learning may introduce dataset shift and non-independence, there is also a risk of introducing bias. For example:

1. If devices are selected for updates when plugged-in or fully charger, then model updates and evaluations computed at different times of the day may be correlated with factors such as day-shift vs. night-shift work schedules.
2. If devices are selected for updates from among the pool of eligible devices at a given time, then devices that are connected at times when few other devices are connected (e.g., night-shift or unusual time zone) may be over-represented in the aggregated output.
3. If selected devices are more likely to have their output kept when the output is computed faster, then a) output from devices with faster processors may be over-represented, with these devices likely newer devices and thus correlated with socioeconomic status; and b) devices with less data may be over-represented, with these devices possibly representing users who use the product less frequently.
4. If the update frequency depends on latency, then certain geographic regions and populations with slower devices or networks may be under-represented.
5. If populations of potential users do not own devices for socioeconomic reasons, they may be under represented in the training dataset, and subsequently also under represented in model training and evaluation.
6. Unweighted aggregation of the model loss across selected devices during federated training may disadvantage model performance on certain devices.

Investigating the degree to which biases in the data-generated process can be identified or mitigated is a crucial problem for both federated learning research and ML research more broadly. Similarly, while limited prior research has demonstrated methods to identify and correct bias in already collected data in the federated setting (e.g. via adversarial methods), further research in this area is needed. Finally, methods for applying post-hoc fairness corrections to models learned from potentially biased training data are also a valuable direction for future work.

Fairness Without Access to Sensitive Attributes

Having nexplicit access to demographic information (race, gender, etc) is critical to many existing fairness criteria. However, the contexts in which federated learning are often deployed also give rise to consisderations of fairness when individual sensitive attributes are not available. For example, this can occur when developing personalized language models or developing fair medical image classifiers without knowing any additional demographic information about individuals. Both measuring and correcting unfairness in contexts where there is no data regarding sensitive group membership is a key area for federated learning research to address especially as only limited research exists in general about examining fairness without access to sensitive attributes.

Other ways to approch this problem include reframing the existing notions of fairness to be equal access to effective models. Under this interpretation of fairness, the goal is to maximize model utility across all individuals, regardless of their (unknown) demographic identities, and regardless of the “goodness” of an individual outcome. This matches the contexts in which federated learning is most commonly used, such as language modeling or medical image classification, where there is no clear notion of an outcome which is “good” for a user, and instead the aim is simply to make correct predictions for users, regardless of the outcome.

Fairness, Privay, and Robustness

Fairness and data privacy seem to be complementary ethical concepts: in many of the real-world contexts where privacy protection is desired, fairness is also desired. Often this is due to the sensitivity of the underlying data. Because federated learning is most likely to be deployed in contexts of sensitive data where both privacy and fairness are desirable, it is important that FL research examines how FL might be able to address existing concerns about fairness in machine learning, and whether FL raises new fairness-related issues.

However, the idea of fairness seems to be in tension with the notions of privacy for which FL seeks to provide guarantess: differentially-private learning typically seeks to obscure individually-identifying characteristics, while fairness often requires knowing individuals’ membership in sensitive groups in order to measure or ensure fair predictions are being made. There has been little work on how (or whether) FL may be able to uniquely address concerns about fairness.

Leveraging Federation to Improve Model Diversity

Federated learning presents an opportunity to leverage uniquely diverse datasets by providing efficient decentralized training protocols along with privacy and non-identifiability guarantees for the resulting models. This means that federated learning enables training on multi-instutitional datasets in many domains where this was previously not possible. This provides a practical opportunity to leverage larger, more diverse datasets and explore the generalizability of models which were previously limited to small populations. More importantly, it provides an opportunity to improve the fairness of these models by combining data across boundaries which are likely to have been correlated with sensitive attributes. For instance, attendance at specific health or educational institutions may be correlated with individuals’ ethnicity or socioeconomic status.