Tag: Governance

Posted on

(Featured) Friendly AI will still be our master. Or, why we should not want to be the pets of super-intelligent computers

Friendly AI will still be our master. Or, why we should not want to be the pets of super-intelligent computers

As our technological capabilities advance at an accelerating pace, so too does the pertinence of the hypothetical conundrum posed by super-intelligent artificial intelligence (AI) and its implications for human freedom. Robert Sparrow examines these implications, drawing extensively from political philosophy and conceptions of agency, and provides an analysis of the societal implications of super-intelligence from a uniquely philosophical standpoint. The author adopts a nuanced perspective, proposing that even benevolent, friendly AI may threaten human freedom in its capability to dominate, consciously or not, its human counterparts. It is this paradox, situated within the broader philosophical discourse of freedom versus domination, that provides the nucleus of this analysis.

The research is grounded in the seminal work of philosopher Philip Pettit, particularly his doctrine of republican freedom. This doctrine centers on the belief that freedom is not merely the absence of interference (negative liberty) but is, critically, the absence of domination or the ability to interfere at will. Pettit famously encapsulated this concept in his metaphor of the “eyeball test,” positing that one is free only when they can look others in the eye without fear or subservience. As we explore the intersection of Pettit’s philosophy and the hypothetical reality of a super-intelligent AI, the profound significance of this test in determining the future of human freedom in a world shared with AI comes sharply into focus.

The “Friendly AI” Problem

Robert Sparrow makes an acute distinction between “friendly” AI and its potential to dominate humanity. The Friendly AI problem stems from the plausible notion that super-intelligent AI, regardless of its benevolence or adherence to human values, may still pose a significant threat to human freedom due to its inherent capacity for domination. A benevolent AI could feasibly operate in a dictatorial manner, modulating its interference in human life based on its determination of human interests. However, a critical distinction must be drawn: a benevolent dictator, even though acting in our interests, is still a dictator. As the author of the article pointedly remarks, to be “free” to act as one wishes only at the behest of another entity, even a well-meaning one, is not true freedom.

Herein lies the crux of the Friendly AI problem: the ability of an AI entity to act in accordance with human interests does not automatically guarantee human freedom. Freedom, as delineated by the republicanism of Pettit, requires resilience; it must not dissolve upon the whims of a more powerful entity. Thus, for the exercise of power by AI to be compatible with human freedom, it must be possible for humans to resist it. One might propose that a genuinely Friendly AI would solicit human input before interfering in our affairs, serving as an efficient executor of our will rather than as a prescriptive entity. Yet, this proposition does not satisfactorily resolve the core tension between AI’s power and freedom and our own. Ultimately, any freedom we might enjoy under a superintelligent AI would be contingent upon the AI’s will, a position which reveals the inherent vulnerability and potential for domination inherent in the Friendly AI concept.

Superintelligence

Bostrom’s notion of Superintelligence, as outlined by Sparrow, posits an AI entity capable of outperforming the best human brains in nearly every economically relevant field. However, the potential domination by such an entity forms the bedrock of the philosophical conflict between benevolence and domination. Drawing on Pettit’s theory of republicanism, it becomes clear that benevolence alone, even if perfectly calibrated to human interests, does not suffice to guarantee freedom. The very ability of a superintelligent AI to interfere unilaterally in human affairs, regardless of its intent, embodies the antithesis of Pettit’s non-domination principle. The analysis further draws attention to the paradox inherent in relying on an external, powerful entity for the regulation of our interests, effectively highlighting the existential risk associated with superintelligent AI. While a superintelligent AI may act in line with human interests, its potential for domination raises questions about the plausibility of achieving a truly “Friendly AI”, a challenge that resonates with the larger discourse on freedom and domination in philosophical studies.

Freedom, Status, and the ‘Eyeball Test’

The question of human freedom in the context of a superintelligent AI intersects with Pettit’s conceptualization of the ‘eyeball test’. In his philosophy, the notion of freedom pivots on the individual’s status within society – a status conferred when one can ‘look others in the eye without reason for fear or deference’. This perspective becomes especially poignant when viewed in the light of a superintelligent entity’s potential dominion. Under such circumstances, the capacity for humans to pass the ‘eyeball test’ could be seriously undermined, as the superintelligent AI, by virtue of its cognitive superiority, could induce both fear and deference. The state of being subjected to the AI’s superior will could consequently impair our ability to ‘look it in the eye’, thereby eroding the human status required for true freedom. This analysis deepens the philosophical understanding of freedom and its inextricable link with status, while simultaneously challenging the concept of a ‘Friendly AI’ from the perspective of republican theory.

The Negative Liberty Doctrine and Technocratic Framing of AI

Berlin’s bifurcation of liberty into negative and positive spheres finds particular resonance in the context of superintelligent AI, and as such, provides a useful framework for interpreting the dominance problem. From a negative liberty perspective – that is, the absence of coercion or interference – the advent of a superintelligent AI could be seen as promoting freedom. However, the technocratic framing of AI, often characterized by an overemphasis on instrumental logic and utility maximization, may inadvertently favor this negative liberty doctrine, potentially to the detriment of positive freedom. This is to say, while an AI’s superior decision-making capabilities could minimize human interference in various spheres of life, it could also inadvertently curtail positive freedom – the opportunity for self-realization and autonomy. As such, this underscores the importance of incorporating broader philosophical considerations into AI research and development, beyond the narrow confines of technocratic perspectives.

This fusion of philosophy and AI research necessitates the introduction of considerations beyond the merely technical and into the sphere of ethics and moral philosophy. The potential for domination by superintelligent AI systems underscores the need for research that specifically targets these concerns, particularly in relation to upholding principles of human dignity, autonomy, and positive freedom. However, achieving this requires a re-evaluation of our current paradigms of AI development that often valorize utility maximization and efficiency. Instead, an approach that truly appreciates the full depth of the challenge must also involve a careful examination of the philosophical underpinnings that inform the design and operation of AI systems. As such, future research in this arena ought to be a collaborative effort between philosophers, ethicists, AI researchers, and policymakers, aimed at defining a coherent set of values and ethical guidelines for the development and use of superintelligent AI.

Abstract

When asked about humanity’s future relationship with computers, Marvin Minsky famously replied “If we’re lucky, they might decide to keep us as pets”. A number of eminent authorities continue to argue that there is a real danger that “super-intelligent” machines will enslave—perhaps even destroy—humanity. One might think that it would swiftly follow that we should abandon the pursuit of AI. Instead, most of those who purport to be concerned about the existential threat posed by AI default to worrying about what they call the “Friendly AI problem”. Roughly speaking this is the question of how we might ensure that the AI that will develop from the first AI that we create will remain sympathetic to humanity and continue to serve, or at least take account of, our interests. In this paper I draw on the “neo-republican” philosophy of Philip Pettit to argue that solving the Friendly AI problem would not change the fact that the advent of super-intelligent AI would be disastrous for humanity by virtue of rendering us the slaves of machines. A key insight of the republican tradition is that freedom requires equality of a certain sort, which is clearly lacking between pets and their owners. Benevolence is not enough. As long as AI has the power to interfere in humanity’s choices, and the capacity to do so without reference to our interests, then it will dominate us and thereby render us unfree. The pets of kind owners are still pets, which is not a status which humanity should embrace. If we really think that there is a risk that research on AI will lead to the emergence of a superintelligence, then we need to think again about the wisdom of researching AI at all.

Friendly AI will still be our master. Or, why we should not want to be the pets of super-intelligent computers
Posted on

(Featured) Predictive policing and algorithmic fairness

Predictive policing and algorithmic fairness

Tzu-Wei Hung and Chun-Ping Yen contribute to the discursive field of predictive policing algorithms (PPAs) and their intersection with structural discrimination. They examine the functioning of PPAs, and lay bare their potential for propagating existing biases in policing practices and thereby question the presumed neutrality of technological interventions in law enforcement. Their investigation underscores the technological manifestation of structural injustices, adding a critical dimension to our understanding of the relationship between modern predictive technologies and societal equity.

An essential aspect of the authors’ argument is the proposition that the root of the problem lies not in the predictive algorithms themselves, but in the biased actions and unjust social structures that shape their application. Their article places this contention within the broader philosophical context, emphasizing the often-overlooked social and political underpinnings of technological systems. Thus, it offers a pertinent contribution to futures studies, prompting a more nuanced understanding of the interplay between (hotly anticipated) advanced technologies like PPAs and the structural realities of societal injustice. The authors provide a robust challenge to deterministic narratives around technology, pointing to the integral role of societal context in determining the impact of predictive policing systems.

Conceptualizing Predictive Policing

Hung and Yen Scrutinize the correlation between data inputs, algorithmic design, and resultant predictions. Their analysis disrupts the popular conception of PPAs as inherently objective and unproblematic, instead illuminating the mechanisms by which structural biases can be inadvertently incorporated and perpetuated through these algorithmic systems. The article’s critical scrutiny of PPAs further elucidates the relational dynamics between data, predictive modeling, and the societal contexts in which they are deployed.

The authors advance the argument that the implications of PPAs extend beyond individual acts of discrimination to reinforce broader systems of structural bias and social injustice. By focusing on the role of PPAs in reproducing existing patterns of discrimination, they elevate the discussion beyond a simplistic focus on technological neutrality or objectivity, situating PPAs within a larger discourse on technological complicity in the perpetuation of social injustices. This perspective fundamentally challenges conventional thinking about PPAs, prompting a shift from an algorithm-centric view to one that acknowledges the socio-political realities that shape and are shaped by these technological systems.

Structural Discrimination, Predictive Policing, and Theoretical Frameworks

The study goes further in its analysis by arguing that discrimination perpetuated through PPAs is, in essence, a manifestation of broader structural discrimination within societal systems. This perspective illuminates the connections between predictive policing and systemic power imbalances, rendering visible the complex ways in which PPAs can reify and intensify existing social injustices. The authors critically underline the potentially negative impact of stakeholder involvement in predictive policing, postulating that equal participation may unintentionally replicate or amplify pre-existing injustices. The analysis posits that the sources of discrimination lie in biased police actions reflecting broader societal inequities rather than the algorithmic systems themselves. Hence, addressing these challenges necessitates a focus not merely on rectifying algorithmic anomalies, but on transforming the unjust structures that they echo.

The authors propose a transformative theoretical framework, referred to as the social safety net schema, which envisions PPAs as integrated within a broader social safety net. This schema reframes the purpose and functioning of PPAs, advocating for their use not to penalize but to predict social vulnerabilities and facilitate requisite assistance. This is a paradigm shift from crime-focused approaches to a welfare-oriented model that situates crime within socio-economic structures. In this schema, the role of predictive policing is reimagined, with crime predictions used as indicators of systemic inequities that necessitate targeted interventions and redistribution of resources. With this reorientation, predictive policing becomes a tool for unveiling societal disparities and assisting in welfare improvement. The implementation of this schema implies a commitment to equity rather than just equality, addressing the nuances and complexities of social realities and aiming at the underlying structures fostering discrimination.

Community and Stakeholder Involvement, and Implications for Future Research

The issue of stakeholder involvement is addressed with both depth and nuance. Acknowledging the criticality of involving diverse stakeholders in the governance and control of predictive policing technology, the authors assert that equal participation could inadvertently reproduce the extant societal disparities. In their view, a stronger representation of underrepresented groups in decision-making processes is vital. This necessitates more resources and mechanisms to ensure their voices are heard and acknowledged in shaping public policies and social structures. The role of local communities in this process is paramount; they act as informed advocates, ensuring the proper understanding and representation of disadvantaged groups. This framework, hence, pivots on a bottom-up approach to power and control over policing, ensuring democratic community control and fostering collective efficacy. The approach is envisioned to counterbalance the persisting inequality, thereby reducing the likelihood of discrimination and improving community control over policing.

The analysis brings forth notable implications for future academic inquiries and policy-making. It endorses the importance of scrutiny of social structures rather than the predictive algorithms themselves as the catalyst for discriminatory practices in predictive policing. This view drives the necessity of further research into the multifaceted intersection between social structures, law enforcement, and advanced predictive technologies. Moreover, it prompts consideration of how policies can be implemented to reflect this understanding, centering on creating a socially aware and equitable technological governance structure. The policy schema of the social safety net for predictive policing, as proposed by the authors, offers a starting point for such a discourse. Future research may focus on implementing and testing this schema, critically examining its effectiveness in mitigating discriminatory impacts of predictive policing, and identifying potential adjustments necessary for enhancing its efficiency and inclusivity. In essence, future inquiries and policy revisions should foster a context-sensitive, democratic, and community-focused approach to predictive policing.

Abstract

This paper examines racial discrimination and algorithmic bias in predictive policing algorithms (PPAs), an emerging technology designed to predict threats and suggest solutions in law enforcement. We first describe what discrimination is in a case study of Chicago’s PPA. We then explain their causes with Broadbent’s contrastive model of causation and causal diagrams. Based on the cognitive science literature, we also explain why fairness is not an objective truth discoverable in laboratories but has context-sensitive social meanings that need to be negotiated through democratic processes. With the above analysis, we next predict why some recommendations given in the bias reduction literature are not as effective as expected. Unlike the cliché highlighting equal participation for all stakeholders in predictive policing, we emphasize power structures to avoid hermeneutical lacunae. Finally, we aim to control PPA discrimination by proposing a governance solution—a framework of a social safety net.

Predictive policing and algorithmic fairness
Posted on

(Featured) We have to talk about emotional AI and crime

We have to talk about emotional AI and crime

Lena Podoletz investigates the utilization of emotional Artificial Intelligence (AI) within the context of law enforcement and criminal justice systems in a critical examination of the sociopolitical, legal, and ethical ramifications of this technology, contextualizing the analysis within the broader landscape of technological trends and potential future applications.

The opening part of the article is devoted to the intricacies of emotion recognition AI, specifically its definition, functionality, and the scientific foundations that inform its development. In dissecting these aspects, the author emphasizes the discrepancy between the common understanding of emotions and the way they are algorithmically conceptualized and processed. Key to this understanding is the recognition that emotional AI, in its current stage of development, relies heavily on theoretical constructs like the ‘basic emotions theory’ and the ‘circumplex model’, the limitations and biases of which can significantly impact its effective and ethical application in law enforcement and criminal justice contexts.

Subsequent sections of the article provide a rigorous evaluation of four areas of concern: accuracy and performance, bias, accountability, and privacy along with other rights and freedoms. The author underscores the need for distinguishing between different uses of emotional AI, stressing that the challenges presented in a law enforcement setting differ significantly from its application in other contexts, such as private homes or smart health environments. This examination extends to issues related to bias in algorithmic decision-making, where existing societal biases can be reproduced and amplified. The complex issue of accountability in emotional AI is also dissected, particularly in terms of attributing responsibility for decisions made by such systems. Finally, the author explores the intersection of emotional AI technologies with privacy and other human rights, indicating that the deployment of these systems can challenge individual autonomy and human dignity.

The thematic concerns presented in the article echo the larger philosophical discourse surrounding the role and implications of AI in society. The author’s evaluation of emotional AI is in line with post-humanist thought, which questions the Cartesian dualism of human and machine, and problematizes the reduction of complex human behaviors and emotions into codified, algorithmic processes. The exploration of bias, accountability, and privacy ties into ongoing debates around the ethics of AI, especially concerning notions of fairness, transparency, and justice in algorithmic decision-making. Moreover, the question of who holds responsibility when AI systems make mistakes or violate rights brings into focus the legal and philosophical concept of moral agency in the age of advanced AI.

Future research might delve deeper into how emotional AI, specifically within law enforcement and criminal justice systems, could be better regulated or standardized to address the highlighted concerns. It would be valuable to explore potential legislative and technical solutions to mitigate bias, improve accuracy, and establish clear lines of accountability. Moreover, further philosophical examination is needed to unpack the implications of emotional AI on our understanding of human emotions, agency, and rights in an increasingly technologized society. Finally, in line with futures studies philosophy, it would be beneficial to conceive of alternative trajectories for the development and deployment of emotional AI that are anchored in ethical foresight and participatory decision-making, thereby ensuring a future that upholds societal well-being and human dignity.

Abstract

Emotional AI is an emerging technology used to make probabilistic predictions about the emotional states of people using data sources, such as facial (micro)-movements, body language, vocal tone or the choice of words. The performance of such systems is heavily debated and so are the underlying scientific methods that serve as the basis for many such technologies. In this article I will engage with this new technology, and with the debates and literature that surround it. Working at the intersection of criminology, policing, surveillance and the study of emotional AI this paper explores and offers a framework of understanding the various issues that these technologies present particularly to liberal democracies. I argue that these technologies should not be deployed within public spaces because there is only a very weak evidence-base as to their effectiveness in a policing and security context, and even more importantly represent a major intrusion to people’s private lives and also represent a worrying extension of policing power because of the possibility that intentions and attitudes may be inferred. Further to this, the danger in the use of such invasive surveillance for the purpose of policing and crime prevention in urban spaces is that it potentially leads to a highly regulated and control-oriented society. I argue that emotion recognition has severe impacts on the right to the city by not only undertaking surveillance of existing situations but also making inferences and probabilistic predictions about future events as well as emotions and intentions.

We have to talk about emotional AI and crime
Posted on

(Featured) Space not for everyone: The problem of social exclusion in the concept of space settlement

Space not for everyone: The problem of social exclusion in the concept of space settlement

Konrad Szocik contests the arguments supporting space colonization and underscores overlooked dimensions of social justice and equity. The primary critique orbits around the arguments of Milan M. Ćirković, who previously dismissed skepticism concerning space colonization, but failed to consider arguments rooted in social justice and equal access. The author points out that the endeavors of space exploration and colonization could inadvertently amplify existing inequalities, transforming these ventures into projects that serve only a fraction of humanity.

The article challenges the comparison Ćirković makes between skepticism about space colonization and hypothetical skepticism about ancestral migrations, arguing that it overlooks the significant disparities between Earth’s physical conditions and those of outer space. Furthermore, the author urges an investigation into the potential impacts of space settlement on equality and access, arguing that the current discourse is dominated by Western perspectives, which may not account for the marginalized and excluded. The author worries that space colonization could simply replicate existing terrestrial injustices, serving only the most privileged while leaving the poorest and most vulnerable behind.

The paper highlights the fear that space settlement, seen as a refuge from Earth’s deteriorating conditions, could be exclusively reserved for the rich or citizens of spacefaring superpowers. This exclusive access could potentially undermine the very purpose of space settlement as a rescue for humanity. Moreover, the author suggests that this enterprise, given the current technical capabilities, might only be realistic for a relatively small number of people. This selectivity questions the moral value of such a venture, particularly if it detracts from efforts to mitigate climate change for the most disadvantaged.

Delving into the philosophical realm, this article brings to the fore the philosophical implications of space settlement, sparking a dialogue reminiscent of John Rawls’ “Theory of Justice”. The highlighted concerns closely echo the principles of fairness and equality in distribution, pointing to a possible “veil of ignorance” in planning space colonization. Similarly, the author’s argument about the unjust distribution of access to space colonization echoes Thomas Pogge’s ideas on global justice and how the actions of some nations can profoundly affect others. This dialogue expands the scope of philosophy and underscores the importance of inclusive ethics in a rapidly advancing technological world.

The discourse of this article presents new pathways for future research in the field of futures studies. Future research could evaluate more inclusive methods of space colonization, investigating alternatives to the currently anticipated elitist selection process. It could also examine the potential of international regulations to ensure equitable access to space resources. Additionally, research could explore the feasibility and ethics of a globally cooperative effort in space colonization. Overall, these directions aim to ensure that the bold ambition of space colonization aligns with the principles of social justice, thereby propelling humanity forward without leaving anyone behind.

Abstract

The subject of this paper is a continuation of the discussion initiated by Milan M. Ćirković. Ćirković criticized a number of arguments skeptical of the idea of space settlement. However, he omitted arguments referring to social justice and equal access, which, as this paper tries to show, are arguably the most serious skeptical remarks against the idea of space colonization. The paper emphasizes that both space exploration and, ultimately, potential space colonization run the risk of exacerbating inequality and, as such, are not projects pursued for all of humanity.

Space not for everyone: The problem of social exclusion in the concept of space settlement
Posted on

(Featured) Autonomous AI Systems in Conflict: Emergent Behavior and Its Impact on Predictability and Reliability

Autonomous AI Systems in Conflict: Emergent Behavior and Its Impact on Predictability and Reliability

Daniel Trusilo investigates the concept of emergent behavior in complex autonomous systems and its implications in dynamic, open context environments such as conflict scenarios. In a nuanced exploration of the intricacies of autonomous systems, the author employs two hypothetical case studies—an intelligence, surveillance, and reconnaissance (ISR) maritime swarm system and a next-generation autonomous humanitarian notification system—to articulate and elucidate the effects of emergent behavior.

In the case of the ISR swarm system, the author underscores how the autonomous algorithm’s unpredictable micro-level behavior can yield reliable macro-level outcomes, enhancing the system’s robustness and resilience against adversarial interventions. Conversely, the humanitarian notification system emphasizes how such systems’ unpredictability can fortify International Humanitarian Law (IHL) compliance, reducing civilian harm, and increasing accountability. Thus, the author emphasizes the dichotomy of emergent behavior: it enhances system reliability and effectiveness while posing novel challenges to predictability and system certification.

Navigating these challenges, the author calls attention to the implications for system certification and ethical interoperability. With the potential for these systems to exhibit unforeseen behavior in actual operations, traditional testing, evaluation, verification, and validation methods seem inadequate. Instead, the author suggests adopting dynamic certification methods, allowing the systems to be continually monitored and adjusted in complex, real-world environments, thereby accommodating emergent behavior. Ethical interoperability, the concurrence of ethical AI principles across different organizations and nations, presents another conundrum, especially with differing ethical guidelines governing AI use in defense.

In its broader philosophical framework, the article contributes to the ongoing discourse on the ethics and morality of AI and autonomous systems, particularly within the realm of futures studies. It underscores the tension between the benefits of autonomous systems and the ethical, moral, and practical challenges they pose. The emergent behavior phenomenon can be seen as a microcosm of the larger issues in AI ethics, reflecting on themes of predictability, control, transparency, and accountability. The navigation of these ethical quandaries implies the need for shared ethical frameworks and standards that can accommodate the complex, unpredictable nature of these systems without compromising the underlying moral principles.

In terms of future research, there are several critical avenues to explore. The implications of emergent behavior in weaponized autonomous systems need careful examination, questioning acceptable risk confidence intervals for such systems’ predictability and reliability. Moreover, the impact of emergent behavior on operator trust and the ongoing issue of machine explainability warrants further exploration. Lastly, it would be pertinent to identify methods of certifying complex autonomous systems while addressing the burgeoning body of distinct, organization-specific ethical AI principles. Such endeavors would help operationalize these principles in light of emergent behavior, thereby contributing to the development of responsible, accountable, and effective AI systems.

Abstract

The development of complex autonomous systems that use artificial intelligence (AI) is changing the nature of conflict. In practice, autonomous systems will be extensively tested before being operationally deployed to ensure system behavior is reliable in expected contexts. However, the complexity of autonomous systems means that they will demonstrate emergent behavior in the open context of real-world conflict environments. This article examines the novel implications of emergent behavior of autonomous AI systems designed for conflict through two case studies. These case studies include (1) a swarm system designed for maritime intelligence, surveillance, and reconnaissance operations, and (2) a next-generation humanitarian notification system. Both case studies represent current or near-future technology in which emergent behavior is possible, demonstrating that such behavior can be both unpredictable and more reliable depending on the level at which the system is considered. This counterintuitive relationship between less predictability and more reliability results in unique challenges for system certification and adherence to the growing body of principles for responsible AI in defense, which must be considered for the real-world operationalization of AI designed for conflict environments.

Autonomous AI Systems in Conflict: Emergent Behavior and Its Impact on Predictability and Reliability
Posted on

(Featured) Algorithmic discrimination in the credit domain: what do we know about it?

Algorithmic discrimination in the credit domain: what do we know about it?

Ana Cristina Bicharra Garcia et al. explore a salient issue in today’s world of ubiquitous artificial intelligence (AI) and machine learning (ML) applications — the intersection of algorithmic decision-making, fairness, and discrimination in the credit domain. Undertaking a systematic literature review from five data sources, the study meticulously categorizes, analyzes, and synthesizes a wide array of existing literature on this topic. Out of an initial 1320 papers identified, 78 were eventually selected for a detailed review.

The research identifies and critically assesses the inherent biases and potential discriminatory practices in algorithmic credit decision systems, particularly regarding race, gender, and other sensitive attributes. A key observation noted is the existing tendency of studies to examine discriminatory effects based on single sensitive attributes. However, the authors highlight the relevance of Kimberlé Crenshaw’s intersectionality theory, which emphasizes the complex layers of discrimination that could emerge when multiple attributes intersect. The study further underscores the issue of ‘reverse redlining’ — a form of discrimination where individuals are either denied credit based on specific attributes or targeted with high-interest loans.

In addition to mapping the landscape of algorithmic fairness and discrimination, the authors offer a critical examination of fairness definitions, technical limitations of fair algorithms, and the challenging equilibrium between data privacy and data sources’ broadening. The authors’ exploration of fairness reveals a lack of consensus on its definition. In fact, the diverse metrics available often lead to contradictory outcomes. Technical actions, the authors assert, have boundaries, and a genuinely discrimination-free environment requires not just fair algorithms, but also structural and societal changes.

In a broader philosophical context, the research paper’s exploration of algorithmic fairness and discrimination in the credit domain harks back to a fundamental question in the philosophy of technology: What is the impact of technology on society and individual human beings? Algorithmic decision-making systems, as exemplified in this research, are not neutral tools; they are imbued with the biases and prejudices of the society they emerge from, raising significant ethical concerns. The credit domain, with its inherent power dynamics and implications on individuals’ livelihoods, serves as a potent illustration of how algorithmic biases can exacerbate societal inequalities. The philosophical debate around the agency of technology, the moral responsibilities of developers and users, and the consequences of technologically mediated discrimination is thereby highly relevant.

As for future research directions, this study presents multiple avenues. A pressing need is the exploration of discrimination scope beyond race, gender, and commonly studied categories. More nuanced understanding of intersectionality in algorithmic discrimination, including the examination of multiple attributes simultaneously, is a vital need. Additionally, further exploration of ‘reverse redlining’, particularly in the Global South, is warranted. A compelling challenge is to arrive at a globally accepted definition of fairness, taking into account the cultural differences that influence societal perceptions. Lastly, the ethical implications of expanding data sources for credit evaluation, while preserving individuals’ privacy, merit in-depth scrutiny. Through these avenues, we can aspire to develop more ethical, fair, and inclusive algorithmic systems, thus addressing the philosophical concerns highlighted above.

Abstract

Many modern digital products use Machine Learning (ML) to emulate human abilities, knowledge, and intellect. In order to achieve this goal, ML systems need the greatest possible quantity of training data to allow the Artificial Intelligence (AI) model to develop an understanding of “what it means to be human”. We propose that the processes by which companies collect this data are problematic, because they entail extractive practices that resemble labour exploitation. The article presents four case studies in which unwitting individuals contribute their humanness to develop AI training sets. By employing a post-Marxian framework, we then analyse the characteristic of these individuals and describe the elements of the capture-machine. Then, by describing and characterising the types of applications that are problematic, we set a foundation for defining and justifying interventions to address this form of labour exploitation.

Algorithmic discrimination in the credit domain: what do we know about it?
Posted on

(Featured) Epistemic diversity and industrial selection bias

Epistemic diversity and industrial selection bias

Manuela Fernández Pinto and Daniel Fernández Pinto offer a compelling examination of the role that funding sources play in shaping scientific consensus, focusing specifically on the influence of private industry. Drawing on the work of Holman and Bruner (2017), the authors use a reinforcement learning model, known as a Q-learning model, to explore industrial selection. The central concept of industrial selection posits that, rather than corrupting individual scientists, private industry can subtly steer scientific outcomes towards their interests by selectively funding research. In the authors’ simulation, three different funding scenarios are considered: research funded solely by industry, research funded solely by a random agent, and research jointly funded by industry and a random agent.

Results from the simulations reinforce the effects of industrial selection observed by Holman and Bruner, showing a divergence from correct scientific hypotheses under sole industry funding. When scientists are funded solely by a random agent, the outcomes are closer to the correct hypothesis. Most notably, when funding is a mix of industry and random allocation, the random element appears to counteract, or at least delay, the bias introduced by industry funding. The authors further observe an unexpected and somewhat paradoxical interaction with methodological diversity, a factor traditionally seen as a strength in scientific communities. Industrial funding effectively exploits this diversity to skew consensus towards industry-friendly outcomes.

The authors then introduce a provocative and potentially contentious suggestion based on their simulations. They propose that a random allocation of funding might be a more effective countermeasure against industrial selection bias than the commonly held belief in meritocratic funding systems, which might inadvertently perpetuate industry bias. This suggestion arises from their observation that a random funding agent in the simulation effectively obstructs industrial selection bias. They also consider the merits and drawbacks of a two-stage random allocation system, wherein only research proposals that pass an initial quality assessment are subject to a subsequent funding lottery.

The study raises compelling philosophical considerations on the influence of funding on the direction and integrity of scientific research. It challenges the common narrative that methodological diversity and meritocracy inherently lead to unbiased, high-quality science, suggesting instead that these can be co-opted by industry to push scientific consensus toward commercially advantageous outcomes. It further incites reflection on the ethical implications of allowing commercial interests to potentially manipulate scientific consensus and the responsibility of society to ensure the pursuit of truth in science. The research also ties into broader discussions on the balance between rational decision-making and randomness, and the potential role of randomness as a mitigating factor in decision-making processes rife with bias or undue influence.

Future research could delve deeper into how a random allocation system might be implemented in practice, particularly regarding the initial quality assessment process. It would also be beneficial to explore how such a system could coexist with traditional funding sources, and what percentage of overall funding would need to be randomly allocated to effectively mitigate industrial selection bias. Additionally, more nuanced simulations could help further untangle the complex relationship between methodological diversity and industrial bias, and identify other possible factors that may be manipulated to sway scientific consensus. Ultimately, this research presents a provocative stepping stone for further exploration into the complex and subtle ways commercial interests may influence scientific research, and potential innovative strategies to counteract such influences.

Abstract

Philosophers of science have argued that epistemic diversity is an asset for the production of scientific knowledge, guarding against the effects of biases, among other advantages. The growing privatization of scientific research, on the contrary, has raised important concerns for philosophers of science, especially with respect to the growing sources of biases in research that it seems to promote. Recently, Holman and Bruner (2017) have shown, using a modified version of Zollman (2010) social network model, that an industrial selection bias can emerge in a scientific community, without corrupting any individual scientist, if the community is epistemically diverse. In this paper, we examine the strength of industrial selection using a reinforcement learning model, which simulates the process of industrial decision-making when allocating funding to scientific projects. Contrary to Holman and Bruner’s model, in which the probability of success of the agents when performing an action is given a priori, in our model the industry learns about the success rate of individual scientists and updates the probability of success on each round. The results of our simulations show that even without previous knowledge of the probability of success of an individual scientist, the industry is still able to disrupt scientific consensus. In fact, the more epistemically diverse the scientific community, the easier it is for the industry to move scientific consensus to the opposite conclusion. Interestingly, our model also shows that having a random funding agent seems to effectively counteract industrial selection bias. Accordingly, we consider the random allocation of funding for research projects as a strategy to counteract industrial selection bias, avoiding commercial exploitation of epistemically diverse communities.

Epistemic diversity and industrial selection bias
Posted on

(Featured) Research Ethics in the Age of Digital Platforms

Research Ethics in the Age of Digital Platforms

José Luis Molina et al. explore the ethical implications of microwork, a novel form of labor facilitated by digital platforms. The authors articulate the nuanced dynamics of this field, focusing primarily on the asymmetrical power relations between microworkers, clients, and platform operators. The piece scrutinizes the transactional nature of microwork, where workers are subject to the platform’s regulations and risk the arbitrary denial of payment or termination of their accounts. Microworkers’ reputation, determined by their prior task success rate, often dictates the quality and quantity of tasks they receive, creating a system of algorithmic governance that perpetuates an exploitative dynamic.

The authors further illustrate this situation by examining the biomedical research standards developed in the aftermath of World War II, which they argue are ill-equipped to address the ethical quandaries posed by microwork. They argue that the conditions of microwork, such as lack of payment floors and the potential for anonymity and segmentation, exacerbate the vulnerability of these workers, aligning them more closely with the exploitation of vulnerable populations in traditional research contexts. They propose a reconceptualization of microworkers as “guest workers” in “digital autocracies,” where the platforms exercise a quasi-governmental control over the working conditions, identity, and compensation of the microworkers.

The authors posit that these digital autocracies extract value through “heteromation” – a process where labor is mediated between cheap human labor and computers, and through the appropriation of workers’ rights to privacy and personal data protection. They argue that microwork platforms, due to their transnational nature and lack of comprehensive regulation, can impose conditions on their workforce that would be unacceptable in traditional employment contexts. They stress the importance of recognizing microworkers as vulnerable populations in research ethics reviews and propose a set of criteria for researchers to ensure the protection of these workers’ rights.

Positioning microwork within the broader philosophical discourse, the authors’ analysis suggests a reevaluation of labor, autonomy, and ethical standards in the digital age. The “digital autocracies” mirror Foucault’s concept of biopower, where power is exerted not merely through coercion but through the management and control of life processes, in this case, the economic existence of microworkers. The situation also reflects Marx’s concept of alienation, as microworkers are distanced from the fruits of their labor, the process of their work, and their fellow workers. The algorithmic governance system also raises questions about agency and autonomy, echoing concerns raised by philosophers such as Hannah Arendt and Jürgen Habermas regarding the instrumentalization of human beings.

Future research in this domain could explore multiple avenues. First, a more extensive empirical study could be conducted to quantify and analyze the conditions of microworkers across different platforms and geographical regions. Second, a comparative study could be undertaken to examine how different regulatory environments impact the working conditions and rights of microworkers. Lastly, a philosophical exploration of notions such as autonomy, justice, and dignity within the digital labor context could provide a more profound understanding of this emerging labor paradigm. The complex interplay of labor, ethics, technology, and globalization, as exemplified by microwork, provides a rich and crucial area for futures studies.

Abstract

Scientific research is growingly increasingly reliant on “microwork” or “crowdsourcing” provided by digital platforms to collect new data. Digital platforms connect clients and workers, charging a fee for an algorithmically managed workflow based on Terms of Service agreements. Although these platforms offer a way to make a living or complement other sources of income, microworkers lack fundamental labor rights and basic safe working conditions, especially in the Global South. We ask how researchers and research institutions address the ethical issues involved in considering microworkers as “human participants.” We argue that current scientific research fails to treat microworkers in the same way as in-person human participants, producing de facto a double morality: one applied to people with rights acknowledged by states and international bodies (e.g., the Helsinki Declaration), the other to guest workers of digital autocracies who have almost no rights at all. We illustrate our argument by drawing on 57 interviews conducted with microworkers in Spanish-speaking countries.

Research Ethics in the Age of Digital Platforms
Posted on

(Featured) Machine learning in bail decisions and judges’ trustworthiness

Machine learning in bail decisions and judges’ trustworthiness

Alexis Morin-Martel navigates the intricate landscape of judicial decision-making and advances the concept of Judge Assistance Systems (JAS), proposing it as a tool for enhancing the trustworthiness of judges in bail decisions. The argument is grounded in the relational theory of procedural justice, which emphasizes the role of trust, voice, neutrality, and respect in the administration of justice. The research underpins its analysis through an exploration of the nuanced terrain of trustworthiness, distinguishing between actual and rich trustworthiness, and articulating the potential role of JAS in amplifying both.

The author leverages the empirical study by Kleinberg et al. (2017a) to illustrate how JAS, equipped with complex algorithms, can assist judges in making more precise bail decisions, thereby enhancing their actual trustworthiness. A key idea espoused is the potential for JAS to act as a check on judicial decision-making, allowing judges to reconsider decisions that deviate significantly from statistical norms. However, the author acknowledges that the implementation of JAS should not undermine the principle of voice, one of the pillars of relational justice, ensuring that defendants have the opportunity to influence the decision-making process.

Further, the study takes into account the perceived trustworthiness of judges when using a JAS. It acknowledges the inherent public skepticism towards algorithmic decisions, often due to their perceived opacity. The argument is made that focusing on accuracy, rather than transparency, of these algorithms is more likely to enhance perceived trustworthiness. Importantly, the author suggests that regular audits within legal institutions could effectively monitor the accuracy of JAS, thus reinforcing public trust over time. However, the author admits that while the ‘voice’ and ‘neutrality’ criteria could likely be met by JAS, its ability to meet the ‘respect’ requirement remains uncertain and needs further examination.

The research article finds a nexus with broader philosophical themes, particularly those concerning human-machine interaction and the ethical implications of algorithmic decision-making. The proposal of JAS as a tool to enhance judicial trustworthiness is reflective of the broader trend towards technocratic governance. This trend raises critical questions about the balance between human judgment and algorithmic precision, and the philosophical implications of delegating traditionally human tasks to artificial intelligence. Moreover, the emphasis on accuracy over transparency in JAS echoes the larger debate on the ethical trade-offs in AI applications, especially in high-stake public decisions.

Future research could explore several intriguing avenues. The extension of JAS to other areas of judicial decision-making, beyond bail decisions, could be considered. Studies could also focus on the development of more transparent and interpretable models without compromising accuracy, addressing public distrust of ‘black box’ algorithms. Furthermore, future research might investigate the potential impact of JAS on other aspects of the relational theory of procedural justice, particularly the ‘respect’ requirement. Lastly, empirical studies evaluating the effectiveness and reliability of JAS in real-world court settings could provide valuable insights into the practicality of implementing such systems.

Abstract

The use of AI algorithms in criminal trials has been the subject of very lively ethical and legal debates recently. While there are concerns over the lack of accuracy and the harmful biases that certain algorithms display, new algorithms seem more promising and might lead to more accurate legal decisions. Algorithms seem especially relevant for bail decisions, because such decisions involve statistical data to which human reasoners struggle to give adequate weight. While getting the right legal outcome is a strong desideratum of criminal trials, advocates of the relational theory of procedural justice give us good reason to think that fairness and perceived fairness of legal procedures have a value that is independent from the outcome. According to this literature, one key aspect of fairness is trustworthiness. In this paper, I argue that using certain algorithms to assist bail decisions could increase three different aspects of judges’ trustworthiness: (1) actual trustworthiness, (2) rich trustworthiness, and (3) perceived trustworthiness.

Machine learning in bail decisions and judges’ trustworthiness
Posted on

(Featured) Bare statistical evidence and the legitimacy of software-based judicial decisions

Bare statistical evidence and the legitimacy of software-based judicial decisions

Eva Schmidt et al. explore the question of whether evidence provided by software systems can serve as a legitimate basis for judicial decisions, focusing on two primary cases: recidivism predictions and DNA cold hit cases. The authors approach this question by analyzing the nature of bare statistical evidence and its relation to individualized evidence. They argue that while bare statistical evidence is generally considered insufficient to meet the standard of proof in criminal and civil cases, software-generated evidence can be individualized and, thus, meet this standard of proof under certain conditions.

In the case of recidivism predictions, the authors discuss the use of software systems such as COMPAS, which rely on bare statistical evidence to estimate the risk of an individual reoffending. They argue that for a sentence to be just and have the potential to serve as an incentive, it must be based on the specific features of the individual concerned, rather than solely on general features of a group that they belong to, which may correlate with high recidivism risk. The authors maintain that bare statistical evidence alone is insufficient for sentencing decisions.

Regarding DNA cold hit cases, the authors propose that statistical evidence generated by software systems like TrueAllele can be individualized through abductive reasoning or inference to the best explanation when it comes to cases of extreme probability. They argue that when the best explanation for the evidence is that the defendant is the source of the crime scene DNA, the evidence can be considered individualized and, thus, meet the standard of proof for criminal cases. This aligns with the normic account of individualized evidence, which posits that the best explanation of a piece of evidence is also the most normal explanation.

The authors’ analysis raises broader philosophical questions concerning the nature of evidence, the role of statistical reasoning in judicial decision-making, and the ethical implications of using software systems in the courtroom. It highlights the importance of distinguishing between different types of support, such as abductive (normic) support and probabilistic support, and of understanding the connections and disconnections between these concepts. Moreover, the paper touches on issues related to transparency, explainability, and fairness in the use of software systems as decision-making aids in the legal context.

Future research could further explore the implications of using software systems for other types of legal evidence and decision-making processes, as well as the ethical and epistemological challenges that these systems pose. Additionally, investigating the relationship between individualized evidence and statistical reasoning could shed light on the nature of evidence itself and the standards of proof required in various legal contexts. Finally, future work could focus on the development of guidelines and best practices for the implementation and evaluation of software systems in the courtroom, addressing issues such as transparency, explainability, and the appropriate weighting of statistical and individualized evidence.

Abstract

Can the evidence provided by software systems meet the standard of proof for civil or criminal cases, and is it individualized evidence? Or, to the contrary, do software systems exclusively provide bare statistical evidence? In this paper, we argue that there are cases in which evidence in the form of probabilities computed by software systems is not bare statistical evidence, and is thus able to meet the standard of proof. First, based on the case of State v. Loomis, we investigate recidivism predictions provided by software systems used in the courtroom. Here, we raise problems for software systems that provide predictions that are based on bare statistical evidence. Second, by examining the case of People v. Chubbs, we argue that the statistical evidence provided by software systems in cold hit DNA cases may in some cases suffice for individualized evidence, on a view on which individualized evidence is evidence that normically supports the relevant proposition (Smith, in Mind 127:1193–1218, 2018).

Bare statistical evidence and the legitimacy of software-based judicial decisions