Text-overlap comparison
This pair was found by matching on title alone (find_title_bucket_dupes.py) — the usual paragraph-embedding similarity search never generated a candidate for it at any threshold. The exhibits below are near-exact word-shingle matches instead — a multi-word verbatim run is the more convincing evidence of shared text than embedding similarity, with the matched span highlighted.
PAPER A
EARLIER¶19-23
Thus, not only is the predictive accuracy of risk assessment tools imperfect, it is also imperfectly presented in the literature. This limited and skewed evidence base creates a risk that decision makers will rely more heavily on risk assessment scores than their accuracy warrants. To mitigate this risk, there is a need for better quality data covering more subpopulations. Validation studies should include more than just one or two performance statistics, and data on the numbers of true and false positives and negatives should be clearly presented. Conflicts of interests need to be disclosed, and reviews by authors with financial conflicts of interests should be treated with caution. In addition to risking over-reliance on risk assessment scores, deficiencies in the evidence base also generate at least three more specific problems, which we explain below: they (i) thwart attempts to match risk assessment tools to different contexts of application, (ii) complicate efforts to determine whether risk assessment tools are unjustifiably discriminatory or stigmatising, and thereby (iii) contribute to the possibility that contentious demographic variables will be prematurely eliminated from assessment tools. 3. The right tool for the context Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used [24]. For example, avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual [25]. The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and
LATER¶33-37
AI FOR RISK ASSESSMENT IN CRIMINAL JUSTICE Artificial Intelligence (AI) is increasingly integrated into risk assessment processes within the criminal justice system, promising to enhance decision-making and resource allocation. The application of AI in risk assessment involves leveraging machine learning algorithms to analyse various factors and predict the likelihood of a defendant reoffending Artificial Intelligence in the Court Justice System 753 or failing to appear in court. While these technologies offer potential benefits, ethical considerations and concerns about fairness and bias must be carefully addressed. Key aspects of AI in risk assessment are stated in the following text. One of the key aspects are predictive algorithms. By this we means that AI algorithms analyse historical data, including criminal records, demographics, and socio-economic factors, to generate risk scores. These scores aim to assist judges and parole boards in making more informed decisions about bail, sentencing, and parole [9]. In second place is individualised risk assessment which represents AI systems striving to provide more individualised risk assessments, moving away from one-size-fits-all approaches. By considering a broader range of factors, including the defendant’s personal history and circumstances, these systems aim to improve the accuracy of risk predictions (Raphael Souza, Amilton & Sperandio Nascimento, 2022). Data-Driven Decision-Making represents AI systems in risk assessment relying heavily on data to identify patterns and correlations. Ensuring the quality and representativeness of the data is crucial to avoid reinforcing existing biases and disparities within the criminal justice system. Transparency and Accountability represents an Ethical deployment of AI in risk assessment, which requires transparency in how algorithms operate and the factors they consider (Douglas, et. al. 2020). Accountability mechanisms must be in place to address concerns related to biased outcomes and the potential impact on individuals, especially in marginalised communities. Human Oversight represents AI systems that can provide valuable insights, though human judgment remains essential. Judges and decision-makers should view AIgenerated risk assessments as tools to inform their decisions rather than as conclusive determinants. Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [Reference Walker23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used (Reference Sinnott-Armstrong, Buzzi, Hyman, Raichle, Kanwisher, Phelps and Morse24). For example, the avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual (Jesper, 2011). 754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate.
EARLIER¶19-24
Thus, not only is the predictive accuracy of risk assessment tools imperfect, it is also imperfectly presented in the literature. This limited and skewed evidence base creates a risk that decision makers will rely more heavily on risk assessment scores than their accuracy warrants. To mitigate this risk, there is a need for better quality data covering more subpopulations. Validation studies should include more than just one or two performance statistics, and data on the numbers of true and false positives and negatives should be clearly presented. Conflicts of interests need to be disclosed, and reviews by authors with financial conflicts of interests should be treated with caution. In addition to risking over-reliance on risk assessment scores, deficiencies in the evidence base also generate at least three more specific problems, which we explain below: they (i) thwart attempts to match risk assessment tools to different contexts of application, (ii) complicate efforts to determine whether risk assessment tools are unjustifiably discriminatory or stigmatising, and thereby (iii) contribute to the possibility that contentious demographic variables will be prematurely eliminated from assessment tools. 3. The right tool for the context Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used [24]. For example, avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual [25]. The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and T. Douglas et al. / European Psychiatry 42 (2017) 134–137 135 https://doi.org/10.1016/j.eurpsy.2016.12.009 Published online by Cambridge University Press
LATER¶33-38
AI FOR RISK ASSESSMENT IN CRIMINAL JUSTICE Artificial Intelligence (AI) is increasingly integrated into risk assessment processes within the criminal justice system, promising to enhance decision-making and resource allocation. The application of AI in risk assessment involves leveraging machine learning algorithms to analyse various factors and predict the likelihood of a defendant reoffending Artificial Intelligence in the Court Justice System 753 or failing to appear in court. While these technologies offer potential benefits, ethical considerations and concerns about fairness and bias must be carefully addressed. Key aspects of AI in risk assessment are stated in the following text. One of the key aspects are predictive algorithms. By this we means that AI algorithms analyse historical data, including criminal records, demographics, and socio-economic factors, to generate risk scores. These scores aim to assist judges and parole boards in making more informed decisions about bail, sentencing, and parole [9]. In second place is individualised risk assessment which represents AI systems striving to provide more individualised risk assessments, moving away from one-size-fits-all approaches. By considering a broader range of factors, including the defendant’s personal history and circumstances, these systems aim to improve the accuracy of risk predictions (Raphael Souza, Amilton & Sperandio Nascimento, 2022). Data-Driven Decision-Making represents AI systems in risk assessment relying heavily on data to identify patterns and correlations. Ensuring the quality and representativeness of the data is crucial to avoid reinforcing existing biases and disparities within the criminal justice system. Transparency and Accountability represents an Ethical deployment of AI in risk assessment, which requires transparency in how algorithms operate and the factors they consider (Douglas, et. al. 2020). Accountability mechanisms must be in place to address concerns related to biased outcomes and the potential impact on individuals, especially in marginalised communities. Human Oversight represents AI systems that can provide valuable insights, though human judgment remains essential. Judges and decision-makers should view AIgenerated risk assessments as tools to inform their decisions rather than as conclusive determinants. Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [Reference Walker23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used (Reference Sinnott-Armstrong, Buzzi, Hyman, Raichle, Kanwisher, Phelps and Morse24). For example, the avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual (Jesper, 2011). 754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives.
EARLIER¶20-24
In addition to risking over-reliance on risk assessment scores, deficiencies in the evidence base also generate at least three more specific problems, which we explain below: they (i) thwart attempts to match risk assessment tools to different contexts of application, (ii) complicate efforts to determine whether risk assessment tools are unjustifiably discriminatory or stigmatising, and thereby (iii) contribute to the possibility that contentious demographic variables will be prematurely eliminated from assessment tools. 3. The right tool for the context Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used [24]. For example, avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual [25]. The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and T. Douglas et al. / European Psychiatry 42 (2017) 134–137 135 https://doi.org/10.1016/j.eurpsy.2016.12.009 Published online by Cambridge University Press
LATER¶34-38
Artificial Intelligence in the Court Justice System 753 or failing to appear in court. While these technologies offer potential benefits, ethical considerations and concerns about fairness and bias must be carefully addressed. Key aspects of AI in risk assessment are stated in the following text. One of the key aspects are predictive algorithms. By this we means that AI algorithms analyse historical data, including criminal records, demographics, and socio-economic factors, to generate risk scores. These scores aim to assist judges and parole boards in making more informed decisions about bail, sentencing, and parole [9]. In second place is individualised risk assessment which represents AI systems striving to provide more individualised risk assessments, moving away from one-size-fits-all approaches. By considering a broader range of factors, including the defendant’s personal history and circumstances, these systems aim to improve the accuracy of risk predictions (Raphael Souza, Amilton & Sperandio Nascimento, 2022). Data-Driven Decision-Making represents AI systems in risk assessment relying heavily on data to identify patterns and correlations. Ensuring the quality and representativeness of the data is crucial to avoid reinforcing existing biases and disparities within the criminal justice system. Transparency and Accountability represents an Ethical deployment of AI in risk assessment, which requires transparency in how algorithms operate and the factors they consider (Douglas, et. al. 2020). Accountability mechanisms must be in place to address concerns related to biased outcomes and the potential impact on individuals, especially in marginalised communities. Human Oversight represents AI systems that can provide valuable insights, though human judgment remains essential. Judges and decision-makers should view AIgenerated risk assessments as tools to inform their decisions rather than as conclusive determinants. Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [Reference Walker23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used (Reference Sinnott-Armstrong, Buzzi, Hyman, Raichle, Kanwisher, Phelps and Morse24). For example, the avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual (Jesper, 2011). 754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives.
EARLIER¶21-25
3. The right tool for the context Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used [24]. For example, avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual [25]. The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and T. Douglas et al. / European Psychiatry 42 (2017) 134–137 135 https://doi.org/10.1016/j.eurpsy.2016.12.009 Published online by Cambridge University Press there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives.
LATER¶35-39
Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [Reference Walker23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used (Reference Sinnott-Armstrong, Buzzi, Hyman, Raichle, Kanwisher, Phelps and Morse24). For example, the avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual (Jesper, 2011). 754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives. Matching risk assessment tools to different contexts of application thus requires trade-offs between positive and negative predictive accuracy. For each context, we must first decide which type of accuracy to prioritise to which degree, and then select a tool that reflects this priority (Suparto, Ellydar, Ardiansyah & Jose, 2023). Unfortunately, in the absence of reliable data, it is not possible to make the latter decision confidently. There is a need for studies using representative samples for relevant subpopulations, avoiding highly selected samples, and presenting performance measures that allow false negative and false positive rates to be reliably estimated for a particular application.
EARLIER¶23-28
The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and T. Douglas et al. / European Psychiatry 42 (2017) 134–137 135 https://doi.org/10.1016/j.eurpsy.2016.12.009 Published online by Cambridge University Press there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives. Matching risk assessment tools to different contexts of application thus requires trade-offs between positive and negative predictive accuracy. For each context, we must first decide which type of accuracy to prioritise to which degree, and then select a tool that reflects this priority. Unfortunately, in the absence of reliable data, it is not possible to make the latter decision confidently. There is a need for studies using representative samples for relevant subpopulations, avoiding highly selected samples, and presenting performance measures that allow false negative and false positive rates to be reliably estimated for a particular application. 4. Discrimination and stigmatisation Some argue that singling out individuals for unfavourable treatment on the basis of their demographic characteristics amounts to unjustified discrimination. This criticism is often levelled at racial profiling by police and airport security [26]. A similar concern might be raised regarding risk assessment tools that take into account an individual’s demographic characteristics such as ethnicity, age, immigration status and gender. It has been suggested that risk assessment tools should employ only ‘individualised’ information, such as information about declared plans and desires based on face to face interviews [17,27], though, even then, judgments may be subject to implicit biases based on the demographic characteristics of the individual being assessed [28]. However, the requirement to utilise only individualised information is overly restrictive. Many would argue that demographic profiling is discriminatory, or problematically so, only when the demographic variables used are recognised social groups (such as ethnic or gender groups) [29], or certain kinds of recognised social groups, for instance, those whose membership is unchosen [30], or that have historically been subject to oppression [31]. Risk assessment tools could theoretically exclude such variables.
LATER¶35-41
Selecting the optimal risk assessment tool for a given application requires trade-offs to be made between false negatives and false positives; attempts to reduce the number of false positives will increase the number of false negatives [Reference Walker23]. Tools with a low rate of false negatives (due to high sensitivity) will be most effective at protecting the public, and may garner most political support, while tools with a low rate of false positives (due to high specificity) will best protect the rights and interests of prisoners and psychiatric patients. The optimal balance between false positives and false negatives is an ethical issue and will depend on the social and political context in which the tool is to be used (Reference Sinnott-Armstrong, Buzzi, Hyman, Raichle, Kanwisher, Phelps and Morse24). For example, the avoidance of false positives may be more important in jurisdictions with less humane detention practices than in jurisdictions with more humane practices, since the less humane the conditions of detention, the greater the harm false positives will tend to impose on the assessed individual (Jesper, 2011). 754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives. Matching risk assessment tools to different contexts of application thus requires trade-offs between positive and negative predictive accuracy. For each context, we must first decide which type of accuracy to prioritise to which degree, and then select a tool that reflects this priority (Suparto, Ellydar, Ardiansyah & Jose, 2023). Unfortunately, in the absence of reliable data, it is not possible to make the latter decision confidently. There is a need for studies using representative samples for relevant subpopulations, avoiding highly selected samples, and presenting performance measures that allow false negative and false positive rates to be reliably estimated for a particular application. Some U.S. jurisdictions use AI systems to augment and, in part, replace judicial discretion in the prediction of the likelihood that an accused (re)offends in the context of criminal bail and sentencing decisions. For example, the Correctional Offender Management Profiling for Alternative Sanctions tool (COMPAS) is used to conduct risk assessment by drawing on the historical data of offenders and analysing that data to produce an output based on the particular offender’s conduct and background. COMPAS integrates 137 responses to a questionnaire, which includes questions ranging from the clearly relevant consideration, ‘How many times has this person been arrested before as an adult or juvenile’, to the more opaque ‘Do you feel discouraged at times’. Importantly, the code and processes underlying COMPAS is secret, and thus not known to the prosecution, defence or judge. COMPAS was developed in 1998, and can be used firstly to predict the likelihood that an accused will fail to appear for trial (the ‘Pretrial Release Risk’ scale), secondly, to predict the Artificial Intelligence in the Court Justice System 755 likelihood that an offender will commit subsequent offences (the ‘General Recidivism’ scale), and thirdly, to predict the likelihood that an offender will commit a violent act in the future (the ‘Violent Recidivism’ scale) (Bell, et. al. 2022). The outcome of each assessment can be used by a court to determine, for example, whether the accused should be released on bail pending trial or be subject to a suspended sentence (recognisance release order) in lieu of a custodial sentence. COMPAS, and risk assessment tools like it, predict the future behaviour of individuals who are either accused of criminal wrongdoing or are incarcerated after having been convicted of a crime. Factors that risk assessment tools might take into account include education and employment, family, socioeconomic and geographical background, and association with convicted criminals by way of family or broader networks. COMPAS has faced a superior court challenge in the U.S. In 2013, Eric Loomis was charged and convicted in relation to a drive-by shooting. The Circuit Court noted that COMPAS had indicated that Loomis had a high risk in each of the pretrial recidivism, general recidivism and violent recidivism scales. On appeal, the Supreme Court of Wisconsin was asked whether the use of the COMPAS tool in sentencing violates a defendant’s right to due process, either because the secret nature of COMPAS prevents defendants from challenging the assessment’s scientific validity, or because COMPAS assessments take gender into account. Justice Bradley, in delivering the reasons of the Court, held that the use of COMPAS by a court was permissible, so long as the judge made the final determination as to the sentence3.
EARLIER¶24-28
T. Douglas et al. / European Psychiatry 42 (2017) 134–137 135 https://doi.org/10.1016/j.eurpsy.2016.12.009 Published online by Cambridge University Press there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives. Matching risk assessment tools to different contexts of application thus requires trade-offs between positive and negative predictive accuracy. For each context, we must first decide which type of accuracy to prioritise to which degree, and then select a tool that reflects this priority. Unfortunately, in the absence of reliable data, it is not possible to make the latter decision confidently. There is a need for studies using representative samples for relevant subpopulations, avoiding highly selected samples, and presenting performance measures that allow false negative and false positive rates to be reliably estimated for a particular application. 4. Discrimination and stigmatisation Some argue that singling out individuals for unfavourable treatment on the basis of their demographic characteristics amounts to unjustified discrimination. This criticism is often levelled at racial profiling by police and airport security [26]. A similar concern might be raised regarding risk assessment tools that take into account an individual’s demographic characteristics such as ethnicity, age, immigration status and gender. It has been suggested that risk assessment tools should employ only ‘individualised’ information, such as information about declared plans and desires based on face to face interviews [17,27], though, even then, judgments may be subject to implicit biases based on the demographic characteristics of the individual being assessed [28]. However, the requirement to utilise only individualised information is overly restrictive. Many would argue that demographic profiling is discriminatory, or problematically so, only when the demographic variables used are recognised social groups (such as ethnic or gender groups) [29], or certain kinds of recognised social groups, for instance, those whose membership is unchosen [30], or that have historically been subject to oppression [31]. Risk assessment tools could theoretically exclude such variables.
LATER¶37-41
754 Ž. Spalević, M. Ilić The appropriate balance between false positives and false negatives will also depend on the stage in the criminal justice process or patient pathway at which the tool will be deployed. For instance, suppose that a risk assessment tool is used to inform decisions about post-sentence detention in a setting where an individual’s initial sentence is proportionate to their degree of responsibility and the seriousness of the crime. Detaining the individual beyond the end of the initial sentence thus involves imposing a disproportionately long period of detention. In this context, special care should be taken to avoid false positives, and there may be grounds to prefer a tool with a very low false positive rate to one that is overall more accurate. However, the situation is different when a tool is used to inform parole decisions. In this context, false positives may lead to refusal of parole and an unnecessarily long period of incarceration from the point of view of public protection. Yet if we assume that the initial sentences are themselves proportionate, then the overall period of detention for ‘false positive’ individuals will remain within the upper limit set by considerations of proportionality. In this context it may be more important to avoid false negatives. Matching risk assessment tools to different contexts of application thus requires trade-offs between positive and negative predictive accuracy. For each context, we must first decide which type of accuracy to prioritise to which degree, and then select a tool that reflects this priority (Suparto, Ellydar, Ardiansyah & Jose, 2023). Unfortunately, in the absence of reliable data, it is not possible to make the latter decision confidently. There is a need for studies using representative samples for relevant subpopulations, avoiding highly selected samples, and presenting performance measures that allow false negative and false positive rates to be reliably estimated for a particular application. Some U.S. jurisdictions use AI systems to augment and, in part, replace judicial discretion in the prediction of the likelihood that an accused (re)offends in the context of criminal bail and sentencing decisions. For example, the Correctional Offender Management Profiling for Alternative Sanctions tool (COMPAS) is used to conduct risk assessment by drawing on the historical data of offenders and analysing that data to produce an output based on the particular offender’s conduct and background. COMPAS integrates 137 responses to a questionnaire, which includes questions ranging from the clearly relevant consideration, ‘How many times has this person been arrested before as an adult or juvenile’, to the more opaque ‘Do you feel discouraged at times’. Importantly, the code and processes underlying COMPAS is secret, and thus not known to the prosecution, defence or judge. COMPAS was developed in 1998, and can be used firstly to predict the likelihood that an accused will fail to appear for trial (the ‘Pretrial Release Risk’ scale), secondly, to predict the Artificial Intelligence in the Court Justice System 755 likelihood that an offender will commit subsequent offences (the ‘General Recidivism’ scale), and thirdly, to predict the likelihood that an offender will commit a violent act in the future (the ‘Violent Recidivism’ scale) (Bell, et. al. 2022). The outcome of each assessment can be used by a court to determine, for example, whether the accused should be released on bail pending trial or be subject to a suspended sentence (recognisance release order) in lieu of a custodial sentence. COMPAS, and risk assessment tools like it, predict the future behaviour of individuals who are either accused of criminal wrongdoing or are incarcerated after having been convicted of a crime. Factors that risk assessment tools might take into account include education and employment, family, socioeconomic and geographical background, and association with convicted criminals by way of family or broader networks. COMPAS has faced a superior court challenge in the U.S. In 2013, Eric Loomis was charged and convicted in relation to a drive-by shooting. The Circuit Court noted that COMPAS had indicated that Loomis had a high risk in each of the pretrial recidivism, general recidivism and violent recidivism scales. On appeal, the Supreme Court of Wisconsin was asked whether the use of the COMPAS tool in sentencing violates a defendant’s right to due process, either because the secret nature of COMPAS prevents defendants from challenging the assessment’s scientific validity, or because COMPAS assessments take gender into account. Justice Bradley, in delivering the reasons of the Court, held that the use of COMPAS by a court was permissible, so long as the judge made the final determination as to the sentence3.