Text-overlap comparison
This pair was found by matching on title alone (find_title_bucket_dupes.py) — the usual paragraph-embedding similarity search never generated a candidate for it at any threshold. The exhibits below are near-exact word-shingle matches instead — a multi-word verbatim run is the more convincing evidence of shared text than embedding similarity, with the matched span highlighted.
PAPER A
PARAGRAPH A¶31-35
P( ˆY = 1 | X = x, S = 0) = P( ˆY = 1 | X = x, S = 1) (2.6) Furthermore, we note that equality of opportunity ([46] or [71]) and avoiding disparate mistreatment [93] are two metrics related to the previous equalized odds, yet weaker. The first one requires only the equality of true positive rates, that is when i = 1 in (2.4), while the second looks at the equality of misclassification error rates across the groups: P( ˆY̸ = Y | S = 0) = P( ˆY̸ = Y | S = 1). (2.7) 1https://www.govinfo.gov/content/pkg/CFR-2017-title29-vol4/xml/CFR-2017-title29-vol4-part1607.xml 5 Thus, equality of odds implies both the lack of disparate mistreatment and equality of opportunity, but not viceversa. Finally, we mention also here predictive parity which was introduced in [20]. It requires the equality of positive predictive values across both groups. Therefore, mathematically it is satisfied when P(Y = 1 | ˆY = 1, S = 0) = P(Y = 1 | ˆY = 1, S = 1). (2.8) The fairness metrics defined above are evaluated only for binary predictions and outcomes. By contrast, we can find also in the literature a set of metrics involving explicit generation of a continuous-valued score denoted here by R ∈[0, 1]. Although scores could be used directly, they can alternatively serve as the input to a thresholding function that outputs a binary prediction. Among this set, we highlight the notion of test-fairness, which extends predictive parity (2.8) when the prediction is a score. An algorithm satisfies this kind of fairness (or it is said to be calibrated) if for all scores r, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. Formally, this is expressed as P(Y = 1 | R = r, S = 0) = P(Y = 1 | R = r, S = 1), for all scores r. This criteria was introduced in [20] and has also been termed as matching conditional frequencies by [46]. A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration.
PARAGRAPH B¶21-25
We note that some authors define predictive parity in a more constrained manner, requiring not only parity for PPV, but also for its counterpart, negative predictive value (NPV), which requires additionally satisfying p(y = 0|ey = 0, G = 0) = p(y = 0|ey = 0, G = 1). Mayson [8] uses the term “overall predictive parity” to describe a predictor with equality across groups in both PPV and NPV, while Berk et al. [4] call this “conditional use accuracy equality”. In the loan example, the model predicted a total of 48 members across both groups to belong to the positive class. However, only 36 out of these were correct predictions; the rest were false positives. Hence, the overall PPV is 36/48 = 0.75. Taken separately, the model predicted a total of 36 members of the orange group and a total of 12 members of the blue group to belong to the positive class. From these predictions, however, only 28 of the orange group and 8 of the blue group were correct. Therefore, the PPV for the orange group is p(y = 1|ey = 1, G = 0) = 28/36 = 7/9 = 0.77 and for the blue group is p(y = 1|ey = 1, G = 1) = 8/12 = 0.66. Since these values differ, the model from our example does not satisfy predictive parity. The fairness metrics discussed above can be evaluated using knowledge only of binary predictions and outcomes. By contrast, we now discuss a set of metrics involving explicit generation of a continuous-valued score s. Optionally, the score can serve as the input to a thresholding function that outputs a binary prediction, though scores can also be used directly, without any thresholding. 2.5 Calibration An algorithm is calibrated if for all scores s, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. [3] [14] Mathematically, this is expressed through p(y = 1|S = s, G = 0) = p(y = 1|S = s, G = 1). This metric has been termed test-fairness by Chouldechova [3], Verma et al. [5], and Mehrabi et al. [12] and as matching conditional frequencies by Hardt et al. [11] There is another related metric termed well-calibration [5] or calibration within groups [2][4] that imposes an additional, more stringent condition. In order for a model to be well-calibrated (or to have calibration within groups), individuals assigned score s must have probability s of belonging to the positive class. If this condition is satisfied, then testfairness will also automatically be satisfied, though the reverse does not hold. The difference between calibration and well-calibration is simply one of mapping; the scores of a calibrated predictor can, using a suitable transformation, be converted to scores satisfying well-calibration.
PARAGRAPH A¶31-35
P( ˆY = 1 | X = x, S = 0) = P( ˆY = 1 | X = x, S = 1) (2.6) Furthermore, we note that equality of opportunity ([46] or [71]) and avoiding disparate mistreatment [93] are two metrics related to the previous equalized odds, yet weaker. The first one requires only the equality of true positive rates, that is when i = 1 in (2.4), while the second looks at the equality of misclassification error rates across the groups: P( ˆY̸ = Y | S = 0) = P( ˆY̸ = Y | S = 1). (2.7) 1https://www.govinfo.gov/content/pkg/CFR-2017-title29-vol4/xml/CFR-2017-title29-vol4-part1607.xml 5 Thus, equality of odds implies both the lack of disparate mistreatment and equality of opportunity, but not viceversa. Finally, we mention also here predictive parity which was introduced in [20]. It requires the equality of positive predictive values across both groups. Therefore, mathematically it is satisfied when P(Y = 1 | ˆY = 1, S = 0) = P(Y = 1 | ˆY = 1, S = 1). (2.8) The fairness metrics defined above are evaluated only for binary predictions and outcomes. By contrast, we can find also in the literature a set of metrics involving explicit generation of a continuous-valued score denoted here by R ∈[0, 1]. Although scores could be used directly, they can alternatively serve as the input to a thresholding function that outputs a binary prediction. Among this set, we highlight the notion of test-fairness, which extends predictive parity (2.8) when the prediction is a score. An algorithm satisfies this kind of fairness (or it is said to be calibrated) if for all scores r, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. Formally, this is expressed as P(Y = 1 | R = r, S = 0) = P(Y = 1 | R = r, S = 1), for all scores r. This criteria was introduced in [20] and has also been termed as matching conditional frequencies by [46]. A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration.
PARAGRAPH B¶21-25
We note that some authors define predictive parity in a more constrained manner, requiring not only parity for PPV, but also for its counterpart, negative predictive value (NPV), which requires additionally satisfying p(y = 0|ey = 0, G = 0) = p(y = 0|ey = 0, G = 1). Mayson [8] uses the term “overall predictive parity” to describe a predictor with equality across groups in both PPV and NPV, while Berk et al. [4] call this “conditional use accuracy equality”. In the loan example, the model predicted a total of 48 members across both groups to belong to the positive class. However, only 36 out of these were correct predictions; the rest were false positives. Hence, the overall PPV is 36/48 = 0.75. Taken separately, the model predicted a total of 36 members of the orange group and a total of 12 members of the blue group to belong to the positive class. From these predictions, however, only 28 of the orange group and 8 of the blue group were correct. Therefore, the PPV for the orange group is p(y = 1|ey = 1, G = 0) = 28/36 = 7/9 = 0.77 and for the blue group is p(y = 1|ey = 1, G = 1) = 8/12 = 0.66. Since these values differ, the model from our example does not satisfy predictive parity. The fairness metrics discussed above can be evaluated using knowledge only of binary predictions and outcomes. By contrast, we now discuss a set of metrics involving explicit generation of a continuous-valued score s. Optionally, the score can serve as the input to a thresholding function that outputs a binary prediction, though scores can also be used directly, without any thresholding. 2.5 Calibration An algorithm is calibrated if for all scores s, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. [3] [14] Mathematically, this is expressed through p(y = 1|S = s, G = 0) = p(y = 1|S = s, G = 1). This metric has been termed test-fairness by Chouldechova [3], Verma et al. [5], and Mehrabi et al. [12] and as matching conditional frequencies by Hardt et al. [11] There is another related metric termed well-calibration [5] or calibration within groups [2][4] that imposes an additional, more stringent condition. In order for a model to be well-calibrated (or to have calibration within groups), individuals assigned score s must have probability s of belonging to the positive class. If this condition is satisfied, then testfairness will also automatically be satisfied, though the reverse does not hold. The difference between calibration and well-calibration is simply one of mapping; the scores of a calibrated predictor can, using a suitable transformation, be converted to scores satisfying well-calibration.
PARAGRAPH A¶32-36
P( ˆY̸ = Y | S = 0) = P( ˆY̸ = Y | S = 1). (2.7) 1https://www.govinfo.gov/content/pkg/CFR-2017-title29-vol4/xml/CFR-2017-title29-vol4-part1607.xml 5 Thus, equality of odds implies both the lack of disparate mistreatment and equality of opportunity, but not viceversa. Finally, we mention also here predictive parity which was introduced in [20]. It requires the equality of positive predictive values across both groups. Therefore, mathematically it is satisfied when P(Y = 1 | ˆY = 1, S = 0) = P(Y = 1 | ˆY = 1, S = 1). (2.8) The fairness metrics defined above are evaluated only for binary predictions and outcomes. By contrast, we can find also in the literature a set of metrics involving explicit generation of a continuous-valued score denoted here by R ∈[0, 1]. Although scores could be used directly, they can alternatively serve as the input to a thresholding function that outputs a binary prediction. Among this set, we highlight the notion of test-fairness, which extends predictive parity (2.8) when the prediction is a score. An algorithm satisfies this kind of fairness (or it is said to be calibrated) if for all scores r, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. Formally, this is expressed as P(Y = 1 | R = r, S = 0) = P(Y = 1 | R = r, S = 1), for all scores r. This criteria was introduced in [20] and has also been termed as matching conditional frequencies by [46]. A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration. Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}.
PARAGRAPH B¶22-26
In the loan example, the model predicted a total of 48 members across both groups to belong to the positive class. However, only 36 out of these were correct predictions; the rest were false positives. Hence, the overall PPV is 36/48 = 0.75. Taken separately, the model predicted a total of 36 members of the orange group and a total of 12 members of the blue group to belong to the positive class. From these predictions, however, only 28 of the orange group and 8 of the blue group were correct. Therefore, the PPV for the orange group is p(y = 1|ey = 1, G = 0) = 28/36 = 7/9 = 0.77 and for the blue group is p(y = 1|ey = 1, G = 1) = 8/12 = 0.66. Since these values differ, the model from our example does not satisfy predictive parity. The fairness metrics discussed above can be evaluated using knowledge only of binary predictions and outcomes. By contrast, we now discuss a set of metrics involving explicit generation of a continuous-valued score s. Optionally, the score can serve as the input to a thresholding function that outputs a binary prediction, though scores can also be used directly, without any thresholding. 2.5 Calibration An algorithm is calibrated if for all scores s, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. [3] [14] Mathematically, this is expressed through p(y = 1|S = s, G = 0) = p(y = 1|S = s, G = 1). This metric has been termed test-fairness by Chouldechova [3], Verma et al. [5], and Mehrabi et al. [12] and as matching conditional frequencies by Hardt et al. [11] There is another related metric termed well-calibration [5] or calibration within groups [2][4] that imposes an additional, more stringent condition. In order for a model to be well-calibrated (or to have calibration within groups), individuals assigned score s must have probability s of belonging to the positive class. If this condition is satisfied, then testfairness will also automatically be satisfied, though the reverse does not hold. The difference between calibration and well-calibration is simply one of mapping; the scores of a calibrated predictor can, using a suitable transformation, be converted to scores satisfying well-calibration. 2.6 Balance for positive/negative class Kleinberg et al. [2] have noted that when the average score s for all individuals constituting the group-specific positive class is the same for both groups of interest, it can be said that there exists balance for the positive class. Similarly, balance for the negative class is satisfied when the average score s for members of the negative class are equal, regardless of group membership. Mathematically this is expressed in terms of expected values. For the negative class, balance requires E[s|y = 0, G = 0] = E[s|y = 0, G = 1], and for the positive class balance requires E[s|y = 1, G = 0] = E[s|y = 1, G = 1].
PARAGRAPH A¶33-37
P(Y = 1 | ˆY = 1, S = 0) = P(Y = 1 | ˆY = 1, S = 1). (2.8) The fairness metrics defined above are evaluated only for binary predictions and outcomes. By contrast, we can find also in the literature a set of metrics involving explicit generation of a continuous-valued score denoted here by R ∈[0, 1]. Although scores could be used directly, they can alternatively serve as the input to a thresholding function that outputs a binary prediction. Among this set, we highlight the notion of test-fairness, which extends predictive parity (2.8) when the prediction is a score. An algorithm satisfies this kind of fairness (or it is said to be calibrated) if for all scores r, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. Formally, this is expressed as P(Y = 1 | R = r, S = 0) = P(Y = 1 | R = r, S = 1), for all scores r. This criteria was introduced in [20] and has also been termed as matching conditional frequencies by [46]. A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration. Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}. 2.3 Relationships between fairness criteria It is also important to note that the wide variety of the proposed criteria formalizing different notions of fairness (see reviews [11] and [89] for more details) has lead sometimes to incompatible formulations. The conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension have been studied in several works [20, 59, 11]. Indeed, in the following Propositions 2.1, 2.2, 2.3 we revisit three impossibility theorems of fairness stating the exclusivity, except in non-degenerate cases, of the three main criteria considered in fair learning.
PARAGRAPH B¶23-27
The fairness metrics discussed above can be evaluated using knowledge only of binary predictions and outcomes. By contrast, we now discuss a set of metrics involving explicit generation of a continuous-valued score s. Optionally, the score can serve as the input to a thresholding function that outputs a binary prediction, though scores can also be used directly, without any thresholding. 2.5 Calibration An algorithm is calibrated if for all scores s, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. [3] [14] Mathematically, this is expressed through p(y = 1|S = s, G = 0) = p(y = 1|S = s, G = 1). This metric has been termed test-fairness by Chouldechova [3], Verma et al. [5], and Mehrabi et al. [12] and as matching conditional frequencies by Hardt et al. [11] There is another related metric termed well-calibration [5] or calibration within groups [2][4] that imposes an additional, more stringent condition. In order for a model to be well-calibrated (or to have calibration within groups), individuals assigned score s must have probability s of belonging to the positive class. If this condition is satisfied, then testfairness will also automatically be satisfied, though the reverse does not hold. The difference between calibration and well-calibration is simply one of mapping; the scores of a calibrated predictor can, using a suitable transformation, be converted to scores satisfying well-calibration. 2.6 Balance for positive/negative class Kleinberg et al. [2] have noted that when the average score s for all individuals constituting the group-specific positive class is the same for both groups of interest, it can be said that there exists balance for the positive class. Similarly, balance for the negative class is satisfied when the average score s for members of the negative class are equal, regardless of group membership. Mathematically this is expressed in terms of expected values. For the negative class, balance requires E[s|y = 0, G = 0] = E[s|y = 0, G = 1], and for the positive class balance requires E[s|y = 1, G = 0] = E[s|y = 1, G = 1]. As Pleiss et al. [15] have explained, this can be viewed as a generalization of the equalized odds metric to non-binary cases. To see this, note that when the score s can take on only the two values 0 and 1, the score itself is the prediction and the term E[s|y = 0, G] then represents the false positive rate and the term E[s|y = 1, G] represents the true positive rate. And, as noted above, under equalized odds the TPR and FPR are equal across groups.
PARAGRAPH A¶35-39
A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration. Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}. 2.3 Relationships between fairness criteria It is also important to note that the wide variety of the proposed criteria formalizing different notions of fairness (see reviews [11] and [89] for more details) has lead sometimes to incompatible formulations. The conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension have been studied in several works [20, 59, 11]. Indeed, in the following Propositions 2.1, 2.2, 2.3 we revisit three impossibility theorems of fairness stating the exclusivity, except in non-degenerate cases, of the three main criteria considered in fair learning. We study first the combination of all three of these metrics and then explore conditions under which it may be possible to simultaneously satisfy two metrics. To begin with, it is interesting to note that from the definition of conditional probability, the respective probability distributions associated with each of these three fairness metrics can be expressed as follows: L(Y, ˆY | S) = L(Y | ˆY , S) × L( ˆY | S) (2.9) = L( ˆY | Y, S) × L(Y | S). (2.10) We observe that on the right-hand side of equality (2.9) the first factor refers to predictive parity, while the second one to statistical parity. Similarly, in the equality (2.10) the first term represents equality of odds while the second one the base rate, that is the distribution of the true target among each group.
PARAGRAPH B¶26-30
2.6 Balance for positive/negative class Kleinberg et al. [2] have noted that when the average score s for all individuals constituting the group-specific positive class is the same for both groups of interest, it can be said that there exists balance for the positive class. Similarly, balance for the negative class is satisfied when the average score s for members of the negative class are equal, regardless of group membership. Mathematically this is expressed in terms of expected values. For the negative class, balance requires E[s|y = 0, G = 0] = E[s|y = 0, G = 1], and for the positive class balance requires E[s|y = 1, G = 0] = E[s|y = 1, G = 1]. As Pleiss et al. [15] have explained, this can be viewed as a generalization of the equalized odds metric to non-binary cases. To see this, note that when the score s can take on only the two values 0 and 1, the score itself is the prediction and the term E[s|y = 0, G] then represents the false positive rate and the term E[s|y = 1, G] represents the true positive rate. And, as noted above, under equalized odds the TPR and FPR are equal across groups. 3 Comparisons and trade-offs between metrics Different fairness metrics formalize varying intuitive notions of fairness. This raises the question of the conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension. 4 28TH JANUARY 2020 We will analyze metrics under the assumption that the “base rate” differs across groups. The base rate of a group is the ratio of people in the group who belong to the positive class (y = 1) to the total number of people in that group. Thus, having non-equal base rates across groups means that p(y = 1|G = 0)̸ = p(y = 1|G = 1). In the subsequent discussion, to simplify the equations we will use the following terms as defined here: • TPRg = p(ey = 1|y = 1, G = g), g ∈{0, 1} — the group-specific true positive rates. • FPRg = p(ey = 1|y = 0, G = g), g ∈{0, 1} — the group-specific false positive rates. • PPVg = p(y = 1|ey = 1, G = g), g ∈{0, 1} — the group-specific positive predictive value.
PARAGRAPH A¶43-47
In the special case of binary classification the result can be sharpened as follows. Observe that we can write for s ∈{0, 1}, P( ˆY = 1 | S = s) = P(Y = 1 | S = s)TPRs + P(Y = 0 | S = s)FPRs (2.11) Then computing the difference between expression (2.11) for each class and assuming that equalized odds holds, namely TPR0 = TPR1 = P( ˆY = 1 | Y = 1) and FPR0 = FPR1 = P( ˆY = 1 | Y = 0), = (P(Y = 1 | S = 0) −P(Y = 1 | S = 1))(P( ˆY = 1 | Y = 1) −P( ˆY = 1 | Y = 0)) Statistical parity requires that left side is exactly zero. Hence, for the right side also being zero necessarily P(Y = 1 | S = 0) = P(Y = 1 | S = 1) or P( ˆY = 1 | Y = 1) = P( ˆY = 1 | Y = 0). However, it is usually assumed that base rates differs across the groups, that is, the ratio of people in the group who belong to the positive class (Y = 1) to the total number of people in that group. Thus, statistical parity and equalized odds are simultaneously achieved only if true and false positive rates are equal. While this is mathematically possible, such condition is not particularly useful since the goal is typically to develop a predictor in which the true positive rate is significantly higher than the false. Proposition 2.2 (Statistical parity vs. Predictive parity) If S is dependent of Y , then either statistical parity holds or predictive parity but not both. By contrast, in the binary classification setup the two fairness metrics are actually simultaneously feasible. Assume that statistical parity holds, that is, P( ˆY = 1|S = 1) = P( ˆY = 1|S = 0) = P( ˆY = 1). Then, from equations (2.9)-(2.10) we can write the difference of positive predictive values
PARAGRAPH B¶27-31
As Pleiss et al. [15] have explained, this can be viewed as a generalization of the equalized odds metric to non-binary cases. To see this, note that when the score s can take on only the two values 0 and 1, the score itself is the prediction and the term E[s|y = 0, G] then represents the false positive rate and the term E[s|y = 1, G] represents the true positive rate. And, as noted above, under equalized odds the TPR and FPR are equal across groups. 3 Comparisons and trade-offs between metrics Different fairness metrics formalize varying intuitive notions of fairness. This raises the question of the conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension. 4 28TH JANUARY 2020 We will analyze metrics under the assumption that the “base rate” differs across groups. The base rate of a group is the ratio of people in the group who belong to the positive class (y = 1) to the total number of people in that group. Thus, having non-equal base rates across groups means that p(y = 1|G = 0)̸ = p(y = 1|G = 1). In the subsequent discussion, to simplify the equations we will use the following terms as defined here: • TPRg = p(ey = 1|y = 1, G = g), g ∈{0, 1} — the group-specific true positive rates. • FPRg = p(ey = 1|y = 0, G = g), g ∈{0, 1} — the group-specific false positive rates. • PPVg = p(y = 1|ey = 1, G = g), g ∈{0, 1} — the group-specific positive predictive value. 3.1 Statistical parity, equalized odds and predictive parity Trade-offs among statistical parity, equalized odds and predictive parity have received significant attention in algorithmic fairness literature under a variety of formulations. Chouldechova [3] articulates the tradeoffs between predictive parity and equalized odds empirically. Kleinberg et al. [2], while not directly referring to these terms, give a closely related result, writing that “the calibration condition and the balance conditions for the positive and negative classes" are “in general incompatible with each other; they can only be simultaneously satisfied in certain highly constrained cases.” Berk et al. [4] discuss incompatibility among metrics by taking several cases and examining the trade-offs in those scenarios.
PARAGRAPH A¶36-40
Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}. 2.3 Relationships between fairness criteria It is also important to note that the wide variety of the proposed criteria formalizing different notions of fairness (see reviews [11] and [89] for more details) has lead sometimes to incompatible formulations. The conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension have been studied in several works [20, 59, 11]. Indeed, in the following Propositions 2.1, 2.2, 2.3 we revisit three impossibility theorems of fairness stating the exclusivity, except in non-degenerate cases, of the three main criteria considered in fair learning. We study first the combination of all three of these metrics and then explore conditions under which it may be possible to simultaneously satisfy two metrics. To begin with, it is interesting to note that from the definition of conditional probability, the respective probability distributions associated with each of these three fairness metrics can be expressed as follows: L(Y, ˆY | S) = L(Y | ˆY , S) × L( ˆY | S) (2.9) = L( ˆY | Y, S) × L(Y | S). (2.10) We observe that on the right-hand side of equality (2.9) the first factor refers to predictive parity, while the second one to statistical parity. Similarly, in the equality (2.10) the first term represents equality of odds while the second one the base rate, that is the distribution of the true target among each group. While the three results for fairness incompatibilities are stated hereafter in a general learning setting and their proofs are gathered in the Appendix 6.1, in this section we present a discussion in the binary classification framework. Let us consider then the following notations for s ∈{0, 1},
PARAGRAPH B¶30-34
In the subsequent discussion, to simplify the equations we will use the following terms as defined here: • TPRg = p(ey = 1|y = 1, G = g), g ∈{0, 1} — the group-specific true positive rates. • FPRg = p(ey = 1|y = 0, G = g), g ∈{0, 1} — the group-specific false positive rates. • PPVg = p(y = 1|ey = 1, G = g), g ∈{0, 1} — the group-specific positive predictive value. 3.1 Statistical parity, equalized odds and predictive parity Trade-offs among statistical parity, equalized odds and predictive parity have received significant attention in algorithmic fairness literature under a variety of formulations. Chouldechova [3] articulates the tradeoffs between predictive parity and equalized odds empirically. Kleinberg et al. [2], while not directly referring to these terms, give a closely related result, writing that “the calibration condition and the balance conditions for the positive and negative classes" are “in general incompatible with each other; they can only be simultaneously satisfied in certain highly constrained cases.” Berk et al. [4] discuss incompatibility among metrics by taking several cases and examining the trade-offs in those scenarios. We offer a common mathematical framework to examine trade-offs among statistical parity, equalized odds and predictive parity. We provide proofs regarding the combination of all three of these metrics and also explore conditions under which it may be possible to simultaneously satisfy two metrics. To provide an initial framing, it is interesting to note that using the basic probability relation p(A, B) = p(A|B)p(B) = p(B|A)p(A), the respective probability distributions associated with each of these three metrics can be expressed as follows: p(y, ey|G) = p(y|ey, G) | {z } Predictive Parity × p(ey|G) | {z } Statistical Parity = p(ey|y, G) | {z } Equalized Odds × p(y|G) | {z } Base Rate (1) 3.1.1 All three? This section considers the feasibility of satisfying all three metrics under the assumption of unequal base rates. We start by assuming a predictor that satisfies both statistical parity and equalized odds, and then examining if it can also satisfy predictive parity. From equation 1, we have:
PARAGRAPH A¶36-40
Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}. 2.3 Relationships between fairness criteria It is also important to note that the wide variety of the proposed criteria formalizing different notions of fairness (see reviews [11] and [89] for more details) has lead sometimes to incompatible formulations. The conditions under which more than one metric can be simultaneously satisfied, and relatedly, the ways in which different metrics might be in tension have been studied in several works [20, 59, 11]. Indeed, in the following Propositions 2.1, 2.2, 2.3 we revisit three impossibility theorems of fairness stating the exclusivity, except in non-degenerate cases, of the three main criteria considered in fair learning. We study first the combination of all three of these metrics and then explore conditions under which it may be possible to simultaneously satisfy two metrics. To begin with, it is interesting to note that from the definition of conditional probability, the respective probability distributions associated with each of these three fairness metrics can be expressed as follows: L(Y, ˆY | S) = L(Y | ˆY , S) × L( ˆY | S) (2.9) = L( ˆY | Y, S) × L(Y | S). (2.10) We observe that on the right-hand side of equality (2.9) the first factor refers to predictive parity, while the second one to statistical parity. Similarly, in the equality (2.10) the first term represents equality of odds while the second one the base rate, that is the distribution of the true target among each group. While the three results for fairness incompatibilities are stated hereafter in a general learning setting and their proofs are gathered in the Appendix 6.1, in this section we present a discussion in the binary classification framework. Let us consider then the following notations for s ∈{0, 1},
PARAGRAPH B¶30-34
In the subsequent discussion, to simplify the equations we will use the following terms as defined here: • TPRg = p(ey = 1|y = 1, G = g), g ∈{0, 1} — the group-specific true positive rates. • FPRg = p(ey = 1|y = 0, G = g), g ∈{0, 1} — the group-specific false positive rates. • PPVg = p(y = 1|ey = 1, G = g), g ∈{0, 1} — the group-specific positive predictive value. 3.1 Statistical parity, equalized odds and predictive parity Trade-offs among statistical parity, equalized odds and predictive parity have received significant attention in algorithmic fairness literature under a variety of formulations. Chouldechova [3] articulates the tradeoffs between predictive parity and equalized odds empirically. Kleinberg et al. [2], while not directly referring to these terms, give a closely related result, writing that “the calibration condition and the balance conditions for the positive and negative classes" are “in general incompatible with each other; they can only be simultaneously satisfied in certain highly constrained cases.” Berk et al. [4] discuss incompatibility among metrics by taking several cases and examining the trade-offs in those scenarios. We offer a common mathematical framework to examine trade-offs among statistical parity, equalized odds and predictive parity. We provide proofs regarding the combination of all three of these metrics and also explore conditions under which it may be possible to simultaneously satisfy two metrics. To provide an initial framing, it is interesting to note that using the basic probability relation p(A, B) = p(A|B)p(B) = p(B|A)p(A), the respective probability distributions associated with each of these three metrics can be expressed as follows: p(y, ey|G) = p(y|ey, G) | {z } Predictive Parity × p(ey|G) | {z } Statistical Parity = p(ey|y, G) | {z } Equalized Odds × p(y|G) | {z } Base Rate (1) 3.1.1 All three? This section considers the feasibility of satisfying all three metrics under the assumption of unequal base rates. We start by assuming a predictor that satisfies both statistical parity and equalized odds, and then examining if it can also satisfy predictive parity. From equation 1, we have:
PARAGRAPH A¶46-51
Proposition 2.2 (Statistical parity vs. Predictive parity) If S is dependent of Y , then either statistical parity holds or predictive parity but not both. By contrast, in the binary classification setup the two fairness metrics are actually simultaneously feasible. Assume that statistical parity holds, that is, P( ˆY = 1|S = 1) = P( ˆY = 1|S = 0) = P( ˆY = 1). Then, from equations (2.9)-(2.10) we can write the difference of positive predictive values PPV0 −PPV1 = TPR0P(Y = 1|S = 0) −TPR1P(Y = 1|S = 1) P( ˆY = 1) (2.12) Under predictive parity the left side of the above equation must be zero, which in turn requires that the ratio of the true positive rates of the two groups be the reciprocal of the ratio of the base rates, namely TPR0 TPR1 = P(Y = 1|S = 1) P(Y = 1|S = 0) (2.13) 7 Thus, while statistical and predictive parity can be simultaneously satisfied even with different base rates, the utility of such a predictor is limited when the ratio of the base rates differs significantly from 1, as this forces the true positive rate for one of the groups to be very low. Proposition 2.3 (Predictive parity vs. Equality of odds) If S is dependent of Y then either predictive parity holds or equality of odds but not both. We explore this incompatibility in more detail in the binary classification framework. If both conditions hold TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1, (2.14) so we can write P( ˆY = 1 | S) = X i=0,1 P( ˆY = 1 | Y = i, S)P( ˆY = 1 | S) = TPR0P(Y = 1 | S) + FPR0P(Y = 0 | S).
PARAGRAPH B¶36-41
p(y = 1|ey = 1, G = 0) −p(y = 1|ey = 1, G = 1) = TPR[p(y = 1|G = 0) −p(y = 1|G = 1)] p(ey = 1) (3) Predictive parity requires that the PPV be equal across both groups, and therefore that the difference on the left side of the above equation be zero, which in turn can only occur when the base rates across the two groups are equal as indicated by the right side of the equation. Note that equalized odds requires both the TPR and the FPR to be the same. However, by demonstrating the incompatibility of just TPR in this section, we provide a sufficient proof for the incompatibility of equalized odds in the given scenario. Thus, when the two groups have unequal base rates, satisfying all three of statistical parity, predictive parity and equalized odds is impossible. This remains true even if the predictor is perfect, as a perfect predictor can not (when the base rates are unequal) satisfy statistical parity. 3.1.2 Statistical parity and predictive parity We now consider conditions under which a predictor can satisfy both statistical and predictive parity. Recall that when statistical parity holds, we have p(ey = 1|G = 0) = p(ey = 1|G = 1) = p(ey = 1). Taking the difference in PPV across the two groups gives: p(y = 1|ey = 1, G = 0) −p(y = 1|ey = 1, G = 1) = TPR0p(y = 1|G = 0) −TPR1p(y = 1|G = 1) p(ey = 1) (4) 5 28TH JANUARY 2020 Under predictive parity the left side of the equation must be zero, which in turn requires that the ratio of the true positive rates of the two groups be the reciprocal of the ratio of the base rates, i.e.: TPR0 TPR1 = p(ey = 1|y = 1, G = 0) p(ey = 1|y = 1, G = 1) = p(y = 1|G = 1) p(y = 1|G = 0) = Base Rate of Group 1 Base Rate of Group 0 (5) Thus, while statistical and predictive parity can be simultaneously satisfied even with different base rates, the utility of such a predictor is limited when the ratio of the base rates differs significantly from 1, as this forces the true positive rate for one of the groups to be very low. As mentioned above, the definition of predictive parity used here, consistent with [3], only requires different groups to have the same PPV. However, if we were to consider the “overall predictive parity” [8], and require a predictor to also have the same NPV across groups, the system would be overconstrained and it would not generally be possible to simultaneously satisfy statistical parity and predictive parity. 3.1.3 Equalized odds and predictive parity Chouldechova (2017) [3] observes that "predictive parity is incompatible with error rate balance when prevalence differs across groups," (p. 5). (As noted above, Chouldechova uses “error rate balance” to describe what we refer to here as equalized odds.) We explore this incompatibility in more detail. As before when equalized odds and predictive parity are satisfied, we have TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1. Noting that
PARAGRAPH A¶47-51
By contrast, in the binary classification setup the two fairness metrics are actually simultaneously feasible. Assume that statistical parity holds, that is, P( ˆY = 1|S = 1) = P( ˆY = 1|S = 0) = P( ˆY = 1). Then, from equations (2.9)-(2.10) we can write the difference of positive predictive values PPV0 −PPV1 = TPR0P(Y = 1|S = 0) −TPR1P(Y = 1|S = 1) P( ˆY = 1) (2.12) Under predictive parity the left side of the above equation must be zero, which in turn requires that the ratio of the true positive rates of the two groups be the reciprocal of the ratio of the base rates, namely TPR0 TPR1 = P(Y = 1|S = 1) P(Y = 1|S = 0) (2.13) 7 Thus, while statistical and predictive parity can be simultaneously satisfied even with different base rates, the utility of such a predictor is limited when the ratio of the base rates differs significantly from 1, as this forces the true positive rate for one of the groups to be very low. Proposition 2.3 (Predictive parity vs. Equality of odds) If S is dependent of Y then either predictive parity holds or equality of odds but not both. We explore this incompatibility in more detail in the binary classification framework. If both conditions hold TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1, (2.14) so we can write P( ˆY = 1 | S) = X i=0,1 P( ˆY = 1 | Y = i, S)P( ˆY = 1 | S) = TPR0P(Y = 1 | S) + FPR0P(Y = 0 | S).
PARAGRAPH B¶37-41
3.1.2 Statistical parity and predictive parity We now consider conditions under which a predictor can satisfy both statistical and predictive parity. Recall that when statistical parity holds, we have p(ey = 1|G = 0) = p(ey = 1|G = 1) = p(ey = 1). Taking the difference in PPV across the two groups gives: p(y = 1|ey = 1, G = 0) −p(y = 1|ey = 1, G = 1) = TPR0p(y = 1|G = 0) −TPR1p(y = 1|G = 1) p(ey = 1) (4) 5 28TH JANUARY 2020 Under predictive parity the left side of the equation must be zero, which in turn requires that the ratio of the true positive rates of the two groups be the reciprocal of the ratio of the base rates, i.e.: TPR0 TPR1 = p(ey = 1|y = 1, G = 0) p(ey = 1|y = 1, G = 1) = p(y = 1|G = 1) p(y = 1|G = 0) = Base Rate of Group 1 Base Rate of Group 0 (5) Thus, while statistical and predictive parity can be simultaneously satisfied even with different base rates, the utility of such a predictor is limited when the ratio of the base rates differs significantly from 1, as this forces the true positive rate for one of the groups to be very low. As mentioned above, the definition of predictive parity used here, consistent with [3], only requires different groups to have the same PPV. However, if we were to consider the “overall predictive parity” [8], and require a predictor to also have the same NPV across groups, the system would be overconstrained and it would not generally be possible to simultaneously satisfy statistical parity and predictive parity. 3.1.3 Equalized odds and predictive parity Chouldechova (2017) [3] observes that "predictive parity is incompatible with error rate balance when prevalence differs across groups," (p. 5). (As noted above, Chouldechova uses “error rate balance” to describe what we refer to here as equalized odds.) We explore this incompatibility in more detail. As before when equalized odds and predictive parity are satisfied, we have TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1. Noting that
PARAGRAPH A¶52-57
This together with equations (2.9)-(2.10) implies P( ˆY = 1|Y = 1, S = 0)P(Y = 1|S = 0) = P(y = 1| ˆY = 1, S = 0) h TPR0P( ˆY = 1|S) + FPR0P(Y = 0|S = 0) i , and using the notations above we obtain TPR0P(Y = 1|S = 0) = PPV0 h TPR0P( ˆY = 1|S) + FPR0(1 −P(Y = 1|S = 0)) i . Finally, we obtain the following expressions for the group-specific base rate for s = 0 P(Y = 1|S = 0) = PPV0FPR0 PPV0FPR0 + (1 −PPV0)TPR0 (2.15) and reasoning likewise for s = 1 P(Y = 1|S = 1) = PPV1FPR1 PPV1FPR1 + (1 −PPV1)TPR1 (2.16) Hence, in the absence of perfect prediction, under assumption (2.14) base rates have to be equal for both equalized odds and predictive parity to simultaneously hold. When perfect prediction is achieved, equations (2.15) and (2.16) take on the indefinite form 0/0 so therefore do not convey anything definitive about base rates in that scenario. We also note that the less strict metric equal opportunity (recall it requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations (2.15) and (2.16) when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between groupspecific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 8 3 Price for fairness in machine learning In this section, we consider how to build fair algorithms and the consequences on the degradation of their performance compared to the possibly unfair case. This corresponds to the price for fairness. Recall that the performance of an algorithm is measured through its risk defined by R(f) = E(l(Y, f(X, S))). Define some class or restriction of classes
PARAGRAPH B¶40-45
As mentioned above, the definition of predictive parity used here, consistent with [3], only requires different groups to have the same PPV. However, if we were to consider the “overall predictive parity” [8], and require a predictor to also have the same NPV across groups, the system would be overconstrained and it would not generally be possible to simultaneously satisfy statistical parity and predictive parity. 3.1.3 Equalized odds and predictive parity Chouldechova (2017) [3] observes that "predictive parity is incompatible with error rate balance when prevalence differs across groups," (p. 5). (As noted above, Chouldechova uses “error rate balance” to describe what we refer to here as equalized odds.) We explore this incompatibility in more detail. As before when equalized odds and predictive parity are satisfied, we have TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1. Noting that But since TPR0 = TPR1, FPR0 = FPR1 and PPV0 = PPV1, equations 7 and 8 will be identical, so base rates for groups 1 and 2 will be the same: p(y = 1|G = 0) = p(y = 1|G = 1). Hence, in the absence of perfect prediction, the base rates have to be equal for both equalized odds and predictive parity to simultaneously hold. When perfect prediction is achieved, equations 7 and 8 take on the indefinite form 0/0 so therefore do not convey anything definitive about base rates in that scenario. We also note that the metric equal opportunity (a less strict counterpart to equalized odds that requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations 7 and 8 when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between group-specific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 6 28TH JANUARY 2020 3.1.4 Equalized odds and statistical parity We now consider if a predictor can simultaneously satisfy equalized odds and statistical parity. As before, for TPR = TPR0 = TPR1 (equal TPR) and FPR = FPR0 = FPR1 (equal FPR): =⇒p(ey = 1|G = 0) −p(ey = 1|G = 1) = (TPR −FPR)[p(y = 1|G = 0) −p(y = 1|G = 1)] (10) Statistical parity requires the left side of equation 10 to be zero. For the equation to hold, this means the right side must also be zero, which can only occur when either TPR = FPR or p(y = 1|G = 0) = p(y = 1|G = 1). The latter case, however, violates the assumption that the base rates are different. Therefore to have both statistical parity and equalized odds, the only possibility is to have TPR = FPR, i.e. the false positive rate and the true positive rate have to be equal. Thus, while simultaneously achieving statistical parity and equalized odds is mathematically possible, it is not particularly useful since the goal is typically to develop a predictor in which the TPR is significantly higher than the FPR.
PARAGRAPH A¶53-57
i , and using the notations above we obtain TPR0P(Y = 1|S = 0) = PPV0 h TPR0P( ˆY = 1|S) + FPR0(1 −P(Y = 1|S = 0)) i . Finally, we obtain the following expressions for the group-specific base rate for s = 0 P(Y = 1|S = 0) = PPV0FPR0 PPV0FPR0 + (1 −PPV0)TPR0 (2.15) and reasoning likewise for s = 1 P(Y = 1|S = 1) = PPV1FPR1 PPV1FPR1 + (1 −PPV1)TPR1 (2.16) Hence, in the absence of perfect prediction, under assumption (2.14) base rates have to be equal for both equalized odds and predictive parity to simultaneously hold. When perfect prediction is achieved, equations (2.15) and (2.16) take on the indefinite form 0/0 so therefore do not convey anything definitive about base rates in that scenario. We also note that the less strict metric equal opportunity (recall it requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations (2.15) and (2.16) when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between groupspecific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 8 3 Price for fairness in machine learning In this section, we consider how to build fair algorithms and the consequences on the degradation of their performance compared to the possibly unfair case. This corresponds to the price for fairness. Recall that the performance of an algorithm is measured through its risk defined by R(f) = E(l(Y, f(X, S))). Define some class or restriction of classes
PARAGRAPH B¶41-45
3.1.3 Equalized odds and predictive parity Chouldechova (2017) [3] observes that "predictive parity is incompatible with error rate balance when prevalence differs across groups," (p. 5). (As noted above, Chouldechova uses “error rate balance” to describe what we refer to here as equalized odds.) We explore this incompatibility in more detail. As before when equalized odds and predictive parity are satisfied, we have TPR0 = TPR1, FPR0 = FPR1, and PPV0 = PPV1. Noting that But since TPR0 = TPR1, FPR0 = FPR1 and PPV0 = PPV1, equations 7 and 8 will be identical, so base rates for groups 1 and 2 will be the same: p(y = 1|G = 0) = p(y = 1|G = 1). Hence, in the absence of perfect prediction, the base rates have to be equal for both equalized odds and predictive parity to simultaneously hold. When perfect prediction is achieved, equations 7 and 8 take on the indefinite form 0/0 so therefore do not convey anything definitive about base rates in that scenario. We also note that the metric equal opportunity (a less strict counterpart to equalized odds that requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations 7 and 8 when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between group-specific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 6 28TH JANUARY 2020 3.1.4 Equalized odds and statistical parity We now consider if a predictor can simultaneously satisfy equalized odds and statistical parity. As before, for TPR = TPR0 = TPR1 (equal TPR) and FPR = FPR0 = FPR1 (equal FPR): =⇒p(ey = 1|G = 0) −p(ey = 1|G = 1) = (TPR −FPR)[p(y = 1|G = 0) −p(y = 1|G = 1)] (10) Statistical parity requires the left side of equation 10 to be zero. For the equation to hold, this means the right side must also be zero, which can only occur when either TPR = FPR or p(y = 1|G = 0) = p(y = 1|G = 1). The latter case, however, violates the assumption that the base rates are different. Therefore to have both statistical parity and equalized odds, the only possibility is to have TPR = FPR, i.e. the false positive rate and the true positive rate have to be equal. Thus, while simultaneously achieving statistical parity and equalized odds is mathematically possible, it is not particularly useful since the goal is typically to develop a predictor in which the TPR is significantly higher than the FPR.
PARAGRAPH A¶39-43
L(Y, ˆY | S) = L(Y | ˆY , S) × L( ˆY | S) (2.9) = L( ˆY | Y, S) × L(Y | S). (2.10) We observe that on the right-hand side of equality (2.9) the first factor refers to predictive parity, while the second one to statistical parity. Similarly, in the equality (2.10) the first term represents equality of odds while the second one the base rate, that is the distribution of the true target among each group. While the three results for fairness incompatibilities are stated hereafter in a general learning setting and their proofs are gathered in the Appendix 6.1, in this section we present a discussion in the binary classification framework. Let us consider then the following notations for s ∈{0, 1}, • PPVs := P(Y = 1 | ˆY = 1, S = s) the group-specific positive predictive values We consider first if a predictor can simultaneously satisfy equalized odds and statistical parity. Proposition 2.1 (Statistical parity vs. Equality of odds) If S is dependent of Y and ˆY is dependent of Y , then either statistical parity holds or equality of odds but not both. In the special case of binary classification the result can be sharpened as follows. Observe that we can write for s ∈{0, 1}, P( ˆY = 1 | S = s) = P(Y = 1 | S = s)TPRs + P(Y = 0 | S = s)FPRs (2.11)
PARAGRAPH B¶42-46
But since TPR0 = TPR1, FPR0 = FPR1 and PPV0 = PPV1, equations 7 and 8 will be identical, so base rates for groups 1 and 2 will be the same: p(y = 1|G = 0) = p(y = 1|G = 1). Hence, in the absence of perfect prediction, the base rates have to be equal for both equalized odds and predictive parity to simultaneously hold. When perfect prediction is achieved, equations 7 and 8 take on the indefinite form 0/0 so therefore do not convey anything definitive about base rates in that scenario. We also note that the metric equal opportunity (a less strict counterpart to equalized odds that requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations 7 and 8 when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between group-specific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 6 28TH JANUARY 2020 3.1.4 Equalized odds and statistical parity We now consider if a predictor can simultaneously satisfy equalized odds and statistical parity. As before, for TPR = TPR0 = TPR1 (equal TPR) and FPR = FPR0 = FPR1 (equal FPR): =⇒p(ey = 1|G = 0) −p(ey = 1|G = 1) = (TPR −FPR)[p(y = 1|G = 0) −p(y = 1|G = 1)] (10) Statistical parity requires the left side of equation 10 to be zero. For the equation to hold, this means the right side must also be zero, which can only occur when either TPR = FPR or p(y = 1|G = 0) = p(y = 1|G = 1). The latter case, however, violates the assumption that the base rates are different. Therefore to have both statistical parity and equalized odds, the only possibility is to have TPR = FPR, i.e. the false positive rate and the true positive rate have to be equal. Thus, while simultaneously achieving statistical parity and equalized odds is mathematically possible, it is not particularly useful since the goal is typically to develop a predictor in which the TPR is significantly higher than the FPR. 3.2 Predictive parity and calibration There has been some confusion in the literature regarding the relationship between these two metrics. Chouldechova [3] has correctly mentioned that “While predictive parity and calibration look like very similar criteria, well-calibrated scores can fail to satisfy predictive parity at a given threshold,” However, in other papers, the discussion on these two metrics sometimes gives less attention to the specifics of how they relate than we think is merited. Given this backdrop, in the present section we explain the difference between predictive parity and calibration with a mathematical derivation and an example.
PARAGRAPH A¶43-47
In the special case of binary classification the result can be sharpened as follows. Observe that we can write for s ∈{0, 1}, P( ˆY = 1 | S = s) = P(Y = 1 | S = s)TPRs + P(Y = 0 | S = s)FPRs (2.11) Then computing the difference between expression (2.11) for each class and assuming that equalized odds holds, namely TPR0 = TPR1 = P( ˆY = 1 | Y = 1) and FPR0 = FPR1 = P( ˆY = 1 | Y = 0), = (P(Y = 1 | S = 0) −P(Y = 1 | S = 1))(P( ˆY = 1 | Y = 1) −P( ˆY = 1 | Y = 0)) Statistical parity requires that left side is exactly zero. Hence, for the right side also being zero necessarily P(Y = 1 | S = 0) = P(Y = 1 | S = 1) or P( ˆY = 1 | Y = 1) = P( ˆY = 1 | Y = 0). However, it is usually assumed that base rates differs across the groups, that is, the ratio of people in the group who belong to the positive class (Y = 1) to the total number of people in that group. Thus, statistical parity and equalized odds are simultaneously achieved only if true and false positive rates are equal. While this is mathematically possible, such condition is not particularly useful since the goal is typically to develop a predictor in which the true positive rate is significantly higher than the false. Proposition 2.2 (Statistical parity vs. Predictive parity) If S is dependent of Y , then either statistical parity holds or predictive parity but not both. By contrast, in the binary classification setup the two fairness metrics are actually simultaneously feasible. Assume that statistical parity holds, that is, P( ˆY = 1|S = 1) = P( ˆY = 1|S = 0) = P( ˆY = 1). Then, from equations (2.9)-(2.10) we can write the difference of positive predictive values
PARAGRAPH B¶43-47
We also note that the metric equal opportunity (a less strict counterpart to equalized odds that requires only equal TPR across groups) is compatible with predictive parity. This is evident from equations 7 and 8 when the condition FPR0 = FPR1 is removed, thereby allowing equalized opportunity and predictive parity to be simultaneously satisfied even with unequal base rates. However, achieving this condition with unequal base rates will require that the FPR differs across the groups. When the difference between the base rates is large, the variation between group-specific FPRs may have to be significant which may reduce suitability for some applications. Hence, while equal opportunity and predictive parity are compatible in the presence of unequal base rates, practitioners should consider the cost (in terms of FPR difference) before attempting to simultaneously achieve both. A similar analysis is possible when we considering parity in negative predictive value instead of positive predictive value, i.e. equal opportunity and parity in NPV are compatible, but only at the cost of variation between group-specific true negative rates (TNRs). 6 28TH JANUARY 2020 3.1.4 Equalized odds and statistical parity We now consider if a predictor can simultaneously satisfy equalized odds and statistical parity. As before, for TPR = TPR0 = TPR1 (equal TPR) and FPR = FPR0 = FPR1 (equal FPR): =⇒p(ey = 1|G = 0) −p(ey = 1|G = 1) = (TPR −FPR)[p(y = 1|G = 0) −p(y = 1|G = 1)] (10) Statistical parity requires the left side of equation 10 to be zero. For the equation to hold, this means the right side must also be zero, which can only occur when either TPR = FPR or p(y = 1|G = 0) = p(y = 1|G = 1). The latter case, however, violates the assumption that the base rates are different. Therefore to have both statistical parity and equalized odds, the only possibility is to have TPR = FPR, i.e. the false positive rate and the true positive rate have to be equal. Thus, while simultaneously achieving statistical parity and equalized odds is mathematically possible, it is not particularly useful since the goal is typically to develop a predictor in which the TPR is significantly higher than the FPR. 3.2 Predictive parity and calibration There has been some confusion in the literature regarding the relationship between these two metrics. Chouldechova [3] has correctly mentioned that “While predictive parity and calibration look like very similar criteria, well-calibrated scores can fail to satisfy predictive parity at a given threshold,” However, in other papers, the discussion on these two metrics sometimes gives less attention to the specifics of how they relate than we think is merited. Given this backdrop, in the present section we explain the difference between predictive parity and calibration with a mathematical derivation and an example. It is possible to view calibration as a generalization of predictive parity to the non-binary setting. The score s discussed in relation to calibration is generally continuous-valued. However, in the special case in which it is limited to the two values 0 and 1 and therefore becomes the prediction itself, achieving calibration is the same as achieving equality across groups in both PPV and NPV. Thus, this satisfies both predictive parity (due to equality across groups in PPV) as well as ”overall predictive parity” (due to equality across groups in both PPV and NPV).
PARAGRAPH A¶32-36
P( ˆY̸ = Y | S = 0) = P( ˆY̸ = Y | S = 1). (2.7) 1https://www.govinfo.gov/content/pkg/CFR-2017-title29-vol4/xml/CFR-2017-title29-vol4-part1607.xml 5 Thus, equality of odds implies both the lack of disparate mistreatment and equality of opportunity, but not viceversa. Finally, we mention also here predictive parity which was introduced in [20]. It requires the equality of positive predictive values across both groups. Therefore, mathematically it is satisfied when P(Y = 1 | ˆY = 1, S = 0) = P(Y = 1 | ˆY = 1, S = 1). (2.8) The fairness metrics defined above are evaluated only for binary predictions and outcomes. By contrast, we can find also in the literature a set of metrics involving explicit generation of a continuous-valued score denoted here by R ∈[0, 1]. Although scores could be used directly, they can alternatively serve as the input to a thresholding function that outputs a binary prediction. Among this set, we highlight the notion of test-fairness, which extends predictive parity (2.8) when the prediction is a score. An algorithm satisfies this kind of fairness (or it is said to be calibrated) if for all scores r, the individuals who have the same score have the same probability of belonging to the positive class, regardless of group membership. Formally, this is expressed as P(Y = 1 | R = r, S = 0) = P(Y = 1 | R = r, S = 1), for all scores r. This criteria was introduced in [20] and has also been termed as matching conditional frequencies by [46]. A related metric called well-calibration [89] or calibration within groups [59] imposes an additional and more stringent condition: a model is well-calibrated if individuals assigned score r must have probability exactly r of belonging to the positive class. If this condition is satisfied, then test-fairness will also hold automatically, though not viceversa. Indeed, we note that the scores of a calibrated predictor can be transformed into scores satisfying well-calibration. Finally, balance for positive/negative class was introduced in [59] as a generalization of the notion of equality of odds. Mathematically, this balance is expressed through the equalities of expected values E(R | Y = i, S = 0) = E(R | Y = i, S = 1), i ∈{0, 1}.
PARAGRAPH B¶53-57
The first column represents the score that the model assigned. In the second and third columns, the numbers outside the parentheses convey the number of people in the group assigned that score. The numbers in parentheses represent the number of people from those assigned that score who actually belong to the positive class. In this example the predictor is calibrated, since given a score, the fraction of people who actually belong to the positive class is independent of the group. For example, for score 0.5, 10/20 = 0.5 = 50% of the people in the orange group with that score and 20/40 = 0.5 = 50% of the people in the blue group with that score belong to the positive class. Does this model satisfy predictive parity? Choosing 0.49 as the threshold gives a total of 60 positive predictions for both the orange group and the blue group. However, of the people with scores greater than 0.49, only 40 members in the orange group and 35 members of the blue group are actually in the positive class, resulting in a PPV of 40/60 = 0.66 for the orange group and 35/60 = 0.583 for the blue group. Thus, while the predictor is calibrated, choosing a threshold of 0.49 does not lead to a set of binary predictions that satisfy predictive parity. It is also interesting to note that if all persons who had a score of 0.25 are instead given a score of 0.4, the model will not only be calibrated (because, as before, people with the same score have the same probability of belonging to the positive class) but also well-calibrated (because, due to this change in scoring, for all scores the score itself would give the probability of belonging to the positive class). However, this change in score would have no impact on the thresholding example above, illustrating that even a well-calibrated model does not, after applying a threshold to produce binary predictions, necessarily satisfy predictive parity. This discussion can be further generalized. Consider the comparison between statistical parity and calibration given by Kleinberg et al. [2]. Kleinberg et al. give an alternate definition of statistical parity which is independent of the choice of threshold, defining it as the condition in which both groups have the same average score. They then prove that when there are unequal base rates and imperfect prediction, statistical parity is incompatible with well-calibration. In one sense, the Kleinberg et al. definition of statistical parity can be understood as a generalization of the binary case. This is because in the special case where the score itself is binary and represents the prediction, the Kleinberg et al. definition reduces to the common (i.e., binary) definition of statistical parity that we have provided in section 2.3. However, when there is both a score s and a binary prediction ey obtained by thresholding the score, the fact that there was statistical parity prior to thresholding in accordance with the definition from Kleinberg et al. does not necessarily imply that there will be statistical parity of the binary predictions in accordance with 2.3. Stated another way, if, prior to thresholding, the average score s is the same across two groups, it does not necessarily follow that thresholding will produce predictions ey in which p(ey|G = 0) = p(ey|G = 1).