Tag: Technology

Posted on

(Featured) Gender bias perpetuation and mitigation in AI technologies: challenges and opportunities

Gender bias perpetuation and mitigation in AI technologies: challenges and opportunities

Sinead O’Connor and Helen Liu investigate a pertinent concern in contemporary artificial intelligence (AI) studies: the manifestation and amplification of gender bias within AI technologies. The authors present a systematic review of multiple case studies which demonstrate the pervasiveness of gender bias across various forms of AI, particularly focusing on textual and visual algorithms. The highlighted studies underscore how AI, far from being an objective tool, can inadvertently perpetuate societal biases ingrained within training datasets, which can extend to controversial societal asymmetries. Moreover, these studies reveal that although de-biasing efforts have been attempted, residual biases often persist due to the depth and complexity of discriminatory patterns.

In an innovative approach, the authors differentiate between bias perpetuation and bias mitigation, exploring this distinction in both text-based and image-based AI contexts. The issue of latent gendered word associations in text is emphasized, wherein researchers strive for a delicate balance between retaining the utility of algorithms and mitigating bias. In image-based AI, the researchers reveal how biases are not only present within algorithms but also entrenched within the evaluative benchmarks themselves. This insight brings into focus the importance of not merely scrutinizing the algorithms but also the standards used to assess their accuracy and bias perpetuation. The researchers also present an incisive critique of the methodological and conceptual issues underlying the treatment of bias in AI research, drawing attention to the often unaddressed question of what counts as ‘bias’ or ‘discrimination’.

The review shifts to an exploration of policy guidelines to address the identified issues, citing initiatives such as the European Commission’s ‘Ethics Guidelines for Trustworthy AI’ and UNESCO’s report on AI and gender equality. These initiatives aim to align AI with fundamental human rights and principles, ensuring their compliance with EU values and norms. The authors conclude with an insightful analysis of the dynamic relationship between gender bias in AI and broader societal structures, highlighting the need for regulatory efforts to manage this interplay.

Placed in a broader philosophical context, the article touches upon several key themes within the philosophy of technology. One of these is the entwined relationship between technology and society. Drawing from scholars like Orlikowski and Bryson, the authors illustrate how AI, as a socio-technical system, is deeply embedded within social structures and reflects societal biases. This notion challenges the conventional perception of technology as neutral and instead, presents it as a socially constructed entity that both shapes and is shaped by society.

The second philosophical theme pertains to the ethics of AI. The authors highlight the necessity of ethical accountability and responsibility in AI development and use. This resonates with the philosophical debates around morality in AI, raising questions about who should be held responsible for algorithmic biases and how should they be held accountable. By proposing cross-disciplinary and accessible approaches in AI research, the authors indirectly invoke the idea of “moral machines” or the notion that AI systems need to be designed with a nuanced understanding of human ethics.

Looking forward, it is essential to deepen the intersectional analysis of bias in AI systems. Future research could expand on the conceptualization and measurement of bias in AI, accounting for the diverse intersections of identities beyond gender, such as race, age, sexuality, and disability. There is also a critical need to explore how AI bias research can engage with non-binary and fluid conceptions of gender to provide a more comprehensive understanding of gender bias.

Abstract

Across the world, artificial intelligence (AI) technologies are being more widely employed in public sector decision-making and processes as a supposedly neutral and an efficient method for optimizing delivery of services. However, the deployment of these technologies has also prompted investigation into the potentially unanticipated consequences of their introduction, to both positive and negative ends. This paper chooses to focus specifically on the relationship between gender bias and AI, exploring claims of the neutrality of such technologies and how its understanding of bias could influence policy and outcomes. Building on a rich seam of literature from both technological and sociological fields, this article constructs an original framework through which to analyse both the perpetuation and mitigation of gender biases, choosing to categorize AI technologies based on whether their input is text or images. Through the close analysis and pairing of four case studies, the paper thus unites two often disparate approaches to the investigation of bias in technology, revealing the large and varied potential for AI to echo and even amplify existing human bias, while acknowledging the important role AI itself can play in reducing or reversing these effects. The conclusion calls for further collaboration between scholars from the worlds of technology, gender studies and public policy in fully exploring algorithmic accountability as well as in accurately and transparently exploring the potential consequences of the introduction of AI technologies.

Gender bias perpetuation and mitigation in AI technologies: challenges and opportunities
Posted on

(Featured) The Ethics of Technology: How Can Indigenous Thought Contribute?

The Ethics of Technology: How Can Indigenous Thought Contribute?

John Weckert and Rogelio Bayod present a comprehensive examination of the intersection between ethics, technology, and Indigenous worldviews. The authors argue that the ethics of technology, which largely remains a peripheral concern in technological developments, could significantly benefit from the incorporation of Indigenous perspectives. They contend that the entrenched paradigms of Western thought, with their focus on materialism, individualism, efficiency, and progress, often marginalize ethical considerations. This, they suggest, is where Indigenous worldviews, which emphasize relationality, spirituality, and a reciprocal relationship with the Earth, could offer a potent alternative.

A key aspect of Indigenous thought highlighted in the paper is the concept of relationality. Indigenous worldviews often consider all entities, living and non-living, as interconnected and mutually influential. This view contrasts with the Western conceptualization of individual entities as distinct and primarily self-interested. Consequently, incorporating this perspective into the ethics of technology could help shift the focus from the maximization of individual benefits to the maintenance of collective well-being. The paper also underscores the Indigenous emphasis on spirituality, where both natural and man-made objects can hold spiritual or non-material significance. This perspective could help challenge the prevailing Western materialistic worldview, fostering a more holistic understanding of technological artifacts and their value.

The authors propose that integrating these Indigenous concepts could provide a foundation for a reimagined Western worldview, even if these elements are interpreted metaphorically rather than literally. Such a worldview, they argue, would not only challenge the prevailing emphasis on materialistic values but could also facilitate a more beneficial development and use of technology. This reframed paradigm would prioritize environmental health, reduce the production of disposable products, and lessen the focus on profitability, efficiency, and individualism. Instead, it would place greater emphasis on care for the Earth, kinship, relationships, and spirituality.

This research contributes to broader philosophical discussions around the ethics of technology and futures studies. It offers a critical reframing of our relationship with technology, drawing on Indigenous worldviews to challenge dominant Western paradigms. By doing so, it highlights the value of diverse perspectives in shaping our technological futures and raises critical questions around the role of values and worldviews in guiding technological development. This paper thus adds to ongoing debates around decolonizing technology and futures studies, and extends them into the sphere of ethics.

The paper suggests numerous avenues for future research. Given its emphasis on the potential of Indigenous worldviews, further explorations could delve deeper into specific Indigenous perspectives on technology, drawing from a wider range of cultures and traditions. Another promising area for future research could involve examining how these Indigenous values could be operationalized within different technological domains, and the possible impacts this could have. Finally, there is a significant need for empirical research on how this paradigm shift might be achieved, and the potential barriers and facilitators involved. This research paper thus opens the door to a rich array of investigations that could fundamentally reshape our understanding of the ethics of technology.

Abstract

The ethics of technology is not as effective as it should. Despite decades of ethical discussion, development and use of new technologies continues apace without much regard to those discussions. Economic and other forces are too powerful. More focus needs to be placed on the values that underpin social attitudes to technology. By seriously looking at Indigenous thought and comparing it with the typical Western way of seeing the world, we can gain a better understanding of our own views. The Indigenous Filipino worldview provides us with a platform for assessing our own core values and suggests modifications to those values. It also indicates ways for broadening and altering the focus of the ethics of technology to make it more effective in helping us to use technologies in ways more conducive to human well-being.

The Ethics of Technology: How Can Indigenous Thought Contribute?
Posted on

(Featured) Can robots be trustworthy?

Can robots be trustworthy?

Ines Schröder et al. present an in-depth exploration of the phenomenological and ethical implications of socially assistive robots (SARs), with a specific focus on their role within the medical sector. Central to the discussion is the concept of responsivity, a construct that the authors argue is inherent to human experience and mirrored, to a certain extent, in human-robot interactions. They explore the nature of this perceived responsivity and its implications for the philosophical understanding of human-robot relations.

The article begins by drawing a distinction between human and artificial responsivity, elucidating the phenomenological structure of human responsivity and how it is translated into SARs’ design. The authors underscore how SARs’ design parameters, such as AI-enhanced speech recognition, physical mobility, and social affordances, culminate in a form of ‘virtual responsivity.’ This virtual responsivity serves to mimic human interaction, creating a semblance of empathy and understanding. However, the authors also emphasize the limitations of this approach, highlighting the potential for deception and the lack of essential direct reciprocity inherent in genuine ethical responsivity.

The crux of the article lies in its examination of the ethical implications of this constructed responsivity. The authors grapple with the potential ethical pitfalls, tensions, and challenges of SARs, particularly within the domain of medical applications. They articulate concerns regarding the preservation of patient autonomy, the balancing of beneficial impact against inherent risks, and the principle of justice in relation to access to advanced technologies. The authors further highlight the three ethically relevant dimensions of vulnerability, dignity, and trust in relation to responsivity, emphasizing the importance of these dimensions in human-robot interactions.

Broadly, the research intersects with larger philosophical themes concerning the nature of consciousness, personhood, and the moral status of non-human entities. The authors’ analysis of SARs’ ‘virtual responsivity’ challenges conventional understandings of these concepts, raising critical questions about the attribution of moral status and the potential for emotional attachment to non-human entities. The exploration of ethical dimensions of vulnerability, dignity, and trust in the context of human-robot interactions further elucidates the evolving dynamics of human-machine relationships, providing a nuanced perspective on the philosophical implications of advanced technology.

Looking towards the future, the research opens several avenues for further exploration. One potential focus is the development of a robust ethical framework for the design and use of SARs, especially in sensitive domains such as healthcare. There is a need for research into ‘ethically sensitive responsiveness,’ which could provide a basis for setting appropriate boundaries in human-robot interactions and ensuring the clear communication of a robot’s capabilities and limitations. Additionally, empirical research exploring the psychological effects of human-robot interactions, particularly in relation to the formation of trust, would be invaluable. Overall, the ethical and philosophical implications of artificial responsivity necessitate a multidisciplinary approach, inviting further dialogue between fields such as robotics, ethics, philosophy, and psychology.

Abstract

Definition of the problem

This article critically addresses the conceptualization of trust in the ethical discussion on artificial intelligence (AI) in the specific context of social robots in care. First, we attempt to define in which respect we can speak of ‘social’ robots and how their ‘social affordances’ affect the human propensity to trust in human–robot interaction. Against this background, we examine the use of the concept of ‘trust’ and ‘trustworthiness’ with respect to the guidelines and recommendations of the High-Level Expert Group on AI of the European Union.

Arguments

Trust is analyzed as a multidimensional concept and phenomenon that must be primarily understood as departing from trusting as a human functioning and capability. To trust is an essential part of the human basic capability to form relations with others. We further want to discuss the concept of responsivity which has been established in phenomenological research as a foundational structure of the relation between the self and the other. We argue that trust and trusting as a capability is fundamentally responsive and needs responsive others to be realized. An understanding of responsivity is thus crucial to conceptualize trusting in the ethical framework of human flourishing. We apply a phenomenological–anthropological analysis to explore the link between certain qualities of social robots that construct responsiveness and thereby simulate responsivity and the human propensity to trust.

Conclusion

Against this background, we want to critically ask whether the concept of trustworthiness in social human–robot interaction could be misguided because of the limited ethical demands that the constructed responsiveness of social robots is able to answer to.

Can robots be trustworthy?
Posted on

(Featured) Beyond the hype: ‘acceptable futures’ for AI and robotic technologies in healthcare

Beyond the hype: ‘acceptable futures’ for AI and robotic technologies in healthcare

Giulia De Togni et al. delve into the complex dynamics of technoscientific expectations surrounding the future of artificial intelligence (AI) and robotic technologies in healthcare. By focusing on surgery, pathology, and social care, they examine the strategies employed by scientists, clinicians, and other stakeholders to navigate and construct visions of an AI-driven future in healthcare. The authors illustrate the challenges faced by these stakeholders, who must balance promissory visions with more realistic expectations, while acknowledging the performative power of high expectations in attracting investment and resources.

The participants in the study engage in a balancing act between high and low expectations, drawing boundaries to maintain credibility for their research and practice while distancing themselves from the hype. They recognize that over-optimistic visions may create false hope and unrealistic expectations of performance, potentially harming AI and robotics research through deflated investment if the outcomes fail to match expectations. The authors demonstrate how the stakeholders negotiate the tension between sustaining and nurturing the hype while calling for the recalibration of expectations within an ethically and socially responsible framework.

Central to the participants’ visions of acceptable futures is the changing nature of human-machine relationships. Through balancing different social, ethical, and technoscientific demands, the participants articulate futures that are perceived as ethically and socially acceptable, as well as realistically achievable. They frame their articulations of both the present and future potential and limitations of AI and robotics technologies within an ethics of expectations that position normative considerations as central to how these expectations are expressed.

This research article contributes to broader philosophical debates concerning the role of expectations and imaginaries in shaping our understanding of technoscientific innovation, human-machine relationships, and the ethics of care. By exploring the dynamic interplay between these factors, the authors shed light on how the future of AI and robotics in healthcare is being constructed and negotiated. This study resonates with key themes in the philosophy of futures studies, including the co-constitution of technological visions and sociotechnical imaginaries, the performativity of expectations, and the ethical dimensions of forecasting and envisioning the future.

To further enrich our understanding of these complex dynamics, future research could explore the perspectives of additional stakeholders, such as patients and policymakers, to gain a more comprehensive picture of the expectations surrounding AI and robotics in healthcare. Additionally, cross-cultural and comparative studies could reveal how different cultural contexts and healthcare systems influence expectations and acceptance of these technologies. Ultimately, by continuing to examine the societal implications of AI and robotic technologies, including their impact on patient autonomy, privacy, and the human aspects of care, scholars can contribute to a more nuanced and ethically responsible vision of the future of healthcare.

Abstract

AI and robotic technologies attract much hype, including utopian and dystopian future visions of technologically driven provision in the health and care sectors. Based on 30 interviews with scientists, clinicians and other stakeholders in the UK, Europe, USA, Australia, and New Zealand, this paper interrogates how those engaged in developing and using AI and robotic applications in health and care characterize their future promise, potential and challenges. We explore the ways in which these professionals articulate and navigate a range of high and low expectations, and promissory and cautionary future visions, around AI and robotic technologies. We argue that, through these articulations and navigations, they construct their own perceptions of socially and ethically ‘acceptable futures’ framed by an ‘ethics of expectations.’ This imbues the envisioned futures with a normative character, articulated in relation to the present context. We build on existing work in the sociology of expectations, aiming to contribute towards better understanding of how technoscientific expectations are navigated and managed by professionals. This is particularly timely since the COVID-19 pandemic gave further momentum to these technologies.

Beyond the hype: ‘acceptable futures’ for AI and robotic technologies in healthcare
Posted on

(Featured) Ethics of AI and Health Care: Towards a Substantive Human Rights Framework

Ethics of AI and Health Care: Towards a Substantive Human Rights Framework

S. Matthew Liao provides an incisive exploration into the ethical considerations intrinsic to the application of artificial intelligence (AI) in healthcare contexts. The paper underscores the burgeoning interest in employing AI for health-related purposes, with AI applications demonstrating competencies in diagnosing certain types of cancer, identifying heart rhythm abnormalities, diagnosing various eye diseases, and even identifying viable embryos. However, the author cautions that the deployment of AI in healthcare settings necessitates adherence to robust ethical frameworks and guidelines.

The author identifies a burgeoning multitude of ethical frameworks for AI that have been proposed over recent years. The count of such frameworks exceeds 80 and stems from a diverse array of sources including private corporations, governmental agencies, academic institutions, and intergovernmental bodies. These frameworks commonly reference the four principles of biomedical ethics: autonomy, beneficence, non-maleficence, and justice, and often include recommendations for transparency, explainability, and trust. However, the author warns that the proliferation of these frameworks has led to confusion, thereby raising pressing questions about the basis, justification, and practical implementation of these recommendations.

In response to this conundrum, the author proposes an AI ethics framework rooted in substantive human rights theory. This proposed framework seeks to address the questions raised by the proliferation of ethical guidelines and to provide clear and practical guidance for the use of AI in healthcare. The author argues for an ethical framework that is not only abstract but also expounds the grounds and justifications of the recommendations it puts forward, as well as how these recommendations should be applied in practice.

The broader philosophical discourse that this research engages with is the ethics of technology and, more specifically, the ethical and moral implications of AI use in healthcare. The central philosophical question the author grapples with is the tension between the rapid development and application of AI in healthcare and the need for substantive ethical guidelines to govern its use. This brings into sharp focus the perennial philosophical tension between progress and ethical constraint, raising the specter of issues such as the nature of autonomy, the definition of harm, and the equitable distribution of benefits and burdens.

For future research, the author’s proposition of a human rights-based ethical framework opens up multiple avenues. First, the application of this framework could be examined in real-world healthcare scenarios to assess its efficacy in guiding ethical AI use. Second, the interplay between this framework and existing legal systems could be studied to ascertain any gaps or overlaps. Lastly, a comparative analysis could be conducted of how this proposed framework fares against other ethical frameworks in use, and how it might be refined or integrated with other approaches for a more robust ethical guidance in healthcare AI applications.

Abstract

There is enormous interest in using artificial intelligence (AI) in health care contexts. But before AI can be used in such settings, we need to make sure that AI researchers and organizations follow appropriate ethical frameworks and guidelines when developing these technologies. In recent years, a great number of ethical frameworks for AI have been proposed. However, these frameworks have tended to be abstract and not explain what grounds and justifies their recommendations and how one should use these recommendations in practice. In this paper, I propose an AI ethics framework that is grounded in substantive, human rights theory and one that can help us address these questions.

Ethics of AI and Health Care: Towards a Substantive Human Rights Framework
Posted on

(Featured) The black box problem revisited. Real and imaginary challenges for automated legal decision making

The black box problem revisited. Real and imaginary challenges for automated legal decision making

Bartosz Brożek et al. explore the ethical and practical dilemmas arising from the integration of Artificial Intelligence (AI) in the realm of law. The authors suggest that despite the perceived opacity and unpredictability of AI, these machines can provide rational and justifiable decisions in legal reasoning. By challenging conventional notions of decision-making and justifiability, the paper reframes the discussion around AI’s role in law and provides a compelling argument for AI’s potential to aid in legal reasoning.

The authors delve into the intricacies of legal decision-making, highlighting the contrast between our traditional expectations and the realities of legal reasoning. They argue that while we expect legal decisions to be based on clearly identifiable structures, algorithmic operations on beliefs, and classical logic, the cognitive science research paints a contrasting picture. The authors further suggest that most legal decisions emerge unconsciously, lack a recognizable structure, and are often influenced by emotional reactions and social training. This observation paves the way for a paradigm shift, suggesting that rather than the process, it is the justifiability of the decision ex post that is paramount.

The authors propose a two-module AI system, one intuitive and the other rational. The intuitive module, powered by machine learning, recognizes patterns from large datasets and makes decisions. The rational module, grounded in logic, does not make decisions but justifies those made by the intuitive module. In this framework, AI can be seen as rational if an acceptable justification can be provided for its decisions, despite their unpredictability. This interesting intertwining of machine learning and logic reshapes our understanding of AI’s role in legal decision-making.

This paper touches upon broader philosophical issues surrounding consciousness, rationality, and decision-making. By arguing for a shift from a process-oriented to a result-oriented evaluation of decision-making, the authors challenge the traditional Kantian perspective. The proposed model, in which an AI’s decisions are assessed based on their post-hoc justifiability, aligns more closely with consequentialist philosophy. This emphasis on the end result rather than the means to reach it further stimulates the ongoing debate on the ethical implications of AI use and the re-evaluation of long-held philosophical tenets in the face of technological advancements.

Future research could explore various facets of this proposed two-module AI system, particularly the interplay and potential conflicts between the intuitive and rational modules. Questions around what constitutes an “acceptable justification” in various legal contexts also demand further exploration. Additionally, research could investigate how this approach to AI in law would intersect with other legal principles, such as fairness, transparency, and due process. Ultimately, the paper presents a compelling case for rethinking the role and evaluation of AI in legal decision-making, opening up intriguing possibilities for future philosophical and legal discourse.

Abstract

This paper addresses the black-box problem in artificial intelligence (AI), and the related problem of explainability of AI in the legal context. We argue, first, that the black box problem is, in fact, a superficial one as it results from an overlap of four different – albeit interconnected – issues: the opacity problem, the strangeness problem, the unpredictability problem, and the justification problem. Thus, we propose a framework for discussing both the black box problem and the explainability of AI. We argue further that contrary to often defended claims the opacity issue is not a genuine problem. We also dismiss the justification problem. Further, we describe the tensions involved in the strangeness and unpredictability problems and suggest some ways to alleviate them.

The black box problem revisited. Real and imaginary challenges for automated legal decision making
Posted on

(Featured) Bare statistical evidence and the legitimacy of software-based judicial decisions

Bare statistical evidence and the legitimacy of software-based judicial decisions

Eva Schmidt et al. explore the question of whether evidence provided by software systems can serve as a legitimate basis for judicial decisions, focusing on two primary cases: recidivism predictions and DNA cold hit cases. The authors approach this question by analyzing the nature of bare statistical evidence and its relation to individualized evidence. They argue that while bare statistical evidence is generally considered insufficient to meet the standard of proof in criminal and civil cases, software-generated evidence can be individualized and, thus, meet this standard of proof under certain conditions.

In the case of recidivism predictions, the authors discuss the use of software systems such as COMPAS, which rely on bare statistical evidence to estimate the risk of an individual reoffending. They argue that for a sentence to be just and have the potential to serve as an incentive, it must be based on the specific features of the individual concerned, rather than solely on general features of a group that they belong to, which may correlate with high recidivism risk. The authors maintain that bare statistical evidence alone is insufficient for sentencing decisions.

Regarding DNA cold hit cases, the authors propose that statistical evidence generated by software systems like TrueAllele can be individualized through abductive reasoning or inference to the best explanation when it comes to cases of extreme probability. They argue that when the best explanation for the evidence is that the defendant is the source of the crime scene DNA, the evidence can be considered individualized and, thus, meet the standard of proof for criminal cases. This aligns with the normic account of individualized evidence, which posits that the best explanation of a piece of evidence is also the most normal explanation.

The authors’ analysis raises broader philosophical questions concerning the nature of evidence, the role of statistical reasoning in judicial decision-making, and the ethical implications of using software systems in the courtroom. It highlights the importance of distinguishing between different types of support, such as abductive (normic) support and probabilistic support, and of understanding the connections and disconnections between these concepts. Moreover, the paper touches on issues related to transparency, explainability, and fairness in the use of software systems as decision-making aids in the legal context.

Future research could further explore the implications of using software systems for other types of legal evidence and decision-making processes, as well as the ethical and epistemological challenges that these systems pose. Additionally, investigating the relationship between individualized evidence and statistical reasoning could shed light on the nature of evidence itself and the standards of proof required in various legal contexts. Finally, future work could focus on the development of guidelines and best practices for the implementation and evaluation of software systems in the courtroom, addressing issues such as transparency, explainability, and the appropriate weighting of statistical and individualized evidence.

Abstract

Can the evidence provided by software systems meet the standard of proof for civil or criminal cases, and is it individualized evidence? Or, to the contrary, do software systems exclusively provide bare statistical evidence? In this paper, we argue that there are cases in which evidence in the form of probabilities computed by software systems is not bare statistical evidence, and is thus able to meet the standard of proof. First, based on the case of State v. Loomis, we investigate recidivism predictions provided by software systems used in the courtroom. Here, we raise problems for software systems that provide predictions that are based on bare statistical evidence. Second, by examining the case of People v. Chubbs, we argue that the statistical evidence provided by software systems in cold hit DNA cases may in some cases suffice for individualized evidence, on a view on which individualized evidence is evidence that normically supports the relevant proposition (Smith, in Mind 127:1193–1218, 2018).

Bare statistical evidence and the legitimacy of software-based judicial decisions
Posted on

(Featured) Algorithmic Nudging: The Need for an Interdisciplinary Oversight

Algorithmic Nudging: The Need for an Interdisciplinary Oversight

Christian Schmauder et al. critically assess the implications and risks of employing “black box” AI systems for the development and implementation of personalized nudges in various domains of life. They begin by outlining the power and promise of algorithmic nudging, drawing attention to how AI-driven nudges could bring about widespread benefits in areas such as health, finance, and sustainability. However, they contend that outsourcing nudging to opaque AI systems poses challenges in terms of understanding the underlying reasons for their effectiveness and addressing potential unintended consequences.

The authors delve deeper into the nuances of algorithmic nudging by examining the role of personalized advice in influencing human decision-making. They highlight a key concern that arises when AI systems attempt to maximize user satisfaction: the tendency of the algorithms to exploit cognitive biases in order to achieve desired outcomes. Consequently, the effectiveness of the AI-developed nudges might come at the cost of truthfulness, ultimately undermining the very goals they were designed to achieve.

To address this issue, the authors advocate for the need to look “under the hood” of AI systems, arguing that understanding the underlying cognitive processes harnessed by these systems is crucial for mitigating unintended side effects. They emphasize the importance of interdisciplinary collaboration between computer scientists, cognitive scientists, and psychologists in the development, monitoring, and refinement of AI systems designed to influence human decision-making.

The authors’ exploration of the limitations and risks of “black box” AI nudges raises broader philosophical concerns, particularly in relation to the ethics of autonomy, transparency, and accountability. These concerns call into question the balance between leveraging AI-driven nudges to benefit society and preserving individual autonomy and freedom of choice. Furthermore, the analysis highlights the tension between relying on AI’s predictive power and fostering a deeper understanding of the mechanisms driving human behavior.

This paper provides a valuable foundation for future research on the ethical and philosophical implications of AI-driven nudging. Further investigation could delve into the possible approaches to designing more transparent and explainable AI systems, exploring how such systems might enhance, rather than hinder, human decision-making processes. Additionally, researchers could examine the moral responsibilities of AI developers and regulators, studying the ethical frameworks necessary to guide the development and deployment of AI nudges that respect human autonomy, values, and dignity. Ultimately, a deeper understanding of these complex philosophical questions will be instrumental in realizing the full potential of AI-driven nudges while safeguarding against their potential pitfalls.

Abstract

Nudge is a popular public policy tool that harnesses well-known biases in human judgement to subtly guide people’s decisions, often to improve their choices or to achieve some socially desirable outcome. Thanks to recent developments in artificial intelligence (AI) methods new possibilities emerge of how and when our decisions can be nudged. On the one hand, algorithmically personalized nudges have the potential to vastly improve human daily lives. On the other hand, blindly outsourcing the development and implementation of nudges to “black box” AI systems means that the ultimate reasons for why such nudges work, that is, the underlying human cognitive processes that they harness, will often be unknown. In this paper, we unpack this concern by considering a series of examples and case studies that demonstrate how AI systems can learn to harness biases in human judgment to reach a specified goal. Drawing on an analogy in a philosophical debate concerning the methodology of economics, we call for the need of an interdisciplinary oversight of AI systems that are tasked and deployed to nudge human behaviours.

Algorithmic Nudging: The Need for an Interdisciplinary Oversight
Posted on

(Featured) Levels of explicability for medical artificial intelligence: What do we normatively need and what can we technically reach?

Levels of explicability for medical artificial intelligence: What do we normatively need and what can we technically reach?

Frank Ursin et al. investigate the ethical considerations associated with medical artificial intelligence (AI), particularly in the context of radiology. They emphasize the importance of implementing explainable AI (XAI) techniques to address epistemic and explanatory concerns that arise when AI is employed in medical decision-making. The authors outline a four-level approach to explicability, comprising disclosure, intelligibility, interpretability, and explainability, with each successive level representing an escalation in the level of detail and clarity provided to the patient or physician.

The authors argue that XAI has great potential in the medical field, and they present two examples from radiology to illustrate its practical applications. The first example involves the use of image inpainting techniques to generate sharper and more detailed saliency maps, which can help localize relevant regions within radiological images. The second example highlights the importance of natural language communication in XAI, where an image-to-text model is used to generate medical reports based on radiological images. These two examples demonstrate that incorporating XAI techniques in radiology can provide valuable insights and improved communication for medical practitioners and patients.

In the paper’s conclusion, the authors emphasize the need for a tailored approach to explicability that considers the needs of patients and the scope of medical decisions. They also advocate for the use of insights gained from medical AI ethics to re-evaluate established medical practices and confront biases in medical classification systems. By applying the four levels of explicability in a thoughtful manner, the authors posit that ethically defensible information processes can be established when utilizing medical AI.

This paper touches on broader philosophical issues related to the ethics of technology, medical autonomy, and the nature of trust in AI-driven decision-making. As AI becomes increasingly integrated into various domains of human activity, questions about transparency, fairness, and the moral implications of AI systems become paramount. This paper demonstrates the necessity of establishing an ethical framework for AI applications in healthcare, providing valuable insights that can be extended to other disciplines as well. By considering the complex interplay between AI-driven systems and human agents, the authors also underscore the importance of understanding how technological advancements impact the broader social fabric and the values we uphold as a society.

Future research in this area could explore the generalizability of the four-level approach to explicability in other medical domains or even non-medical contexts. Additionally, researchers may investigate how the incorporation of diverse perspectives in the development of AI systems and explainability techniques can mitigate the potential for biases and discriminatory outcomes. It would also be valuable to study how XAI can be adapted to the specific needs and preferences of individual patients or physicians, creating personalized approaches to explicability. Lastly, researchers may wish to assess the long-term impact of integrating XAI in medical practice, particularly in terms of patient satisfaction, physician trust, and overall quality of care.

Abstract

Definition of the problem

The umbrella term “explicability” refers to the reduction of opacity of artificial intelligence (AI) systems. These efforts are challenging for medical AI applications because higher accuracy often comes at the cost of increased opacity. This entails ethical tensions because physicians and patients desire to trace how results are produced without compromising the performance of AI systems. The centrality of explicability within the informed consent process for medical AI systems compels an ethical reflection on the trade-offs. Which levels of explicability are needed to obtain informed consent when utilizing medical AI?

Arguments

We proceed in five steps: First, we map the terms commonly associated with explicability as described in the ethics and computer science literature, i.e., disclosure, intelligibility, interpretability, and explainability. Second, we conduct a conceptual analysis of the ethical requirements for explicability when it comes to informed consent. Third, we distinguish hurdles for explicability in terms of epistemic and explanatory opacity. Fourth, this then allows to conclude the level of explicability physicians must reach and what patients can expect. In a final step, we show how the identified levels of explicability can technically be met from the perspective of computer science. Throughout our work, we take diagnostic AI systems in radiology as an example.

Conclusion

We determined four levels of explicability that need to be distinguished for ethically defensible informed consent processes and showed how developers of medical AI can technically meet these requirements.

Levels of explicability for medical artificial intelligence: What do we normatively need and what can we technically reach?
Posted on

(Featured) A phenomenological perspective on AI ethical failures: The case of facial recognition technology

A phenomenological perspective on AI ethical failures: The case of facial recognition technology

Yuni Wen and Matthias Holweg conduct a philosophical analysis of the responses of four prominent technology firms to the ethical concerns surrounding the use and development of facial recognition technology. The article meticulously delves into the controversies surrounding Amazon, IBM, Microsoft, and Google, as they grapple with public backlash and stakeholder disapproval. By analyzing these cases, the authors elucidate four distinct strategies that these organizations employ to mitigate potential reputation loss: deflection, improvement, validation, and pre-emption. They astutely highlight the spectrum of these responses, ranging from the most accommodative to the most defensive approach.

The authors propose three possible antecedents that may determine an organization’s response strategy to controversial AI technology: the financial importance of the technology to the company, the strategic importance of the technology to the company’s product and service offerings, and the degree to which the controversial technology violates the company’s stated public values. Through their examination of the facial recognition controversies and the strategies employed by the tech giants, they provide invaluable insights into how these factors contribute to shaping the responses of companies facing ethical dilemmas in AI technology.

Although the article’s primary focus is on large technology firms, it acknowledges the limitations of its analysis and encourages further research on small and medium-sized firms, non-profit organizations, public sector organizations, and other entities that may intentionally misuse AI for nefarious purposes. It also highlights the need for future research to consider the interplay between organizational strategies and the varying global regulatory landscape concerning AI technology, given the diverse policy initiatives and regional differences.

The article not only contributes to the ongoing discourse about AI ethics but also resonates with broader philosophical debates on corporate social responsibility and the role of organizations in shaping a just and equitable society. In an era of unprecedented technological advances and heightened awareness of ethical concerns, this research raises pertinent questions about the duties and responsibilities that companies bear in addressing the potential social and moral implications of their products and services. It underscores the challenge that organizations face in balancing financial interests and strategic goals with ethical imperatives and societal expectations.

To enrich our understanding of the complex interplay between organizations and AI ethics, future research could explore the processes through which companies develop and implement their response strategies, with an emphasis on the role of leadership, organizational culture, and internal and external stakeholder dynamics. Moreover, investigating how these strategies evolve over time and assessing their effectiveness in addressing public concerns could provide valuable insights into best practices for organizations navigating the ethical minefield of AI technology. Ultimately, this line of inquiry would contribute significantly to our understanding of how corporations can foster the responsible development and use of AI, ensuring that its potential benefits are realized while mitigating its ethical risks.

Abstract

As more and more companies adopt artificial intelligence to increase the efficiency and effectiveness of their products and services, they expose themselves to ethical crises and potentially damaging public controversy associated with its use. Despite the prevalence of AI ethical problems, most companies are strategically unprepared to respond effectively to the public. This paper aims to advance our empirical understanding of company responses to AI ethical crises by focusing on the rise and fall of facial recognition technology. Specifically, through a comparative case study of how four big technology companies responded to public outcry over their facial recognition programs, we not only demonstrated the unfolding and consequences of public controversies over this new technology, but also identified and described four major types of company responses—Deflection, Improvement, Validation, and Pre-emption. These findings pave the way for future research on the management of controversial technology and the ethics of AI.

A phenomenological perspective on AI ethical failures: The case of facial recognition technology