Software vulnerability due to practical drift

The proliferation of information and communication technologies (ICTs) into all aspects of life poses unique ethical challenges as our modern societies become increasingly dependent on the flawless operation of these technologies. As we increasingly entrust our privacy, our well-being and our lives to an ever greater number of computers we need to look more closely at the risks and ethical implications of these developments. By emphasising the vulnerability of software and the practice of professional software developers, we want to make clear the ethical aspects of producing potentially flawed software. This paper outlines some of the vulnerabilities associated with software systems and identifies a number of social and organisational factors affecting software developers and contributing to these vulnerabilities. Scott A. Snook’s theory of practical drift is used as the basis for our analysis. We show that this theory, originally developed to explain the failure of a military organisation, can be used to understand how professional software developers “drift away” from procedures and processes designed to ensure quality and prevent software vulnerability. Based on interviews with software developers in two Norwegian companies we identify two areas where social factors compel software developers to drift away from a global set of rules constituting software development processes and methods. Issues of pleasure and control and difference in mental models contribute to an uncoupling from established practices designed to guarantee the reliability of software and thus diminish its vulnerability.     

科学家可否从事军事科学研究？

科学家可否从事军事科学研究？本文在陈述两种对立观点的理由以及科学家面临两难选择的基础上,认为唯一可行的方案也许是：科学家按照自己的良心行事,根据具体情况决定自己何去何从。最后讨论了从事军事科学研究的科学家应该承担的道德义务。     

The ethics of information transparency

The paper investigates the ethics of information transparency (henceforth transparency). It argues that transparency is not an ethical principle in itself but a pro-ethical condition for enabling or impairing other ethical practices or principles. A new definition of transparency is offered in order to take into account the dynamics of information production and the differences between data and information. It is then argued that the proposed definition provides a better understanding of what sort of information should be disclosed and what sort of information should be used in order to implement and make effective the ethical practices and principles to which an organisation is committed. The concepts of “heterogeneous organisation” and “autonomous computational artefact” are further defined in order to clarify the ethical implications of the technology used in implementing information transparency. It is argued that explicit ethical designs, which describe how ethical principles are embedded into the practice of software design, would represent valuable information that could be disclosed by organisations in order to support their ethical standing.     

科技伦理视角下的人工智能研究

人工智能在深刻影响人类社会生产生活方式的同时,也引发诸多伦理困境与挑战,建立新的科技伦理规范以推动人工智能更好服务人类,成为全社会共同关注的主题。本文从科技伦理的视角,围绕机器人、算法、大数据、无人驾驶等人工智能领域所出现的伦理主体、责任分担、技术安全、歧视与公平性、隐私与数据保护等问题,以及人工智能技术的伦理治理,对国内外相关研究成果进行回顾分析,并提出未来需要在中国情境下伦理原则与治理体系的建立、人工智能伦理研究的跨学科合作、理论分析与实践案例的融合、多元主体伦理角色分工与协作等方面进行进一步研究。     

Identity,profiling algorithms and a world of ambient intelligence

The tendency towards an increasing integration of the informational web into our daily physical world (in particular in so-called Ambient Intelligent technologies which combine ideas derived from the field of Ubiquitous Computing, Intelligent User Interfaces and Ubiquitous Communication) is likely to make the development of successful profiling and personalization algorithms, like the ones currently used by internet companies such as Amazon, even more important than it is today. I argue that the way in which we experience ourselves necessarily goes through a moment of technical mediation. Because of this algorithmic profiling that thrives on continuous reconfiguration of identification should not be understood as a supplementary process which maps a pre-established identity that exists independently from the profiling practice. In order to clarify how the experience of one’s identity can become affected by such machine-profiling a theoretical exploration of identity is made (including Agamben’s understanding of an apparatus, Ricoeur’s distinction between idem- and ipse-identity, and Stiegler’s notion of a conjunctive–disjunctive relationship towards retentional apparatuses). Although it is clear that no specific predictions about the impact of Ambient Intelligent technologies can be made without taking more particulars into account, the theoretical concepts are used to describe three general scenarios about the way wherein the experience of identity might become affected. To conclude, I argue that the experience of one’s identity may affect whether the cases of unwarranted discrimination resulting from ubiquitous differentiations and identifications within an Ambient Intelligent environment, will become a matter of societal concern.     

负责任创新框架下的人工智能伦理问题研究

负责任创新(RRI)旨在科学和技术创新过程中社会行动者和创新者彼此负责,同时会考虑创新过程中的伦理问题及创新产品的可接受性、可持续性和社会需求。其内涵分为行政定义和学术定义。在预期、反思、协商、反应、可持续性、关怀这6种负责任创新框架的基础上引入无私利性,基于负责任创新这7种框架理念解析人工智能技术发展带来的安全、隐私、结果等三方面主要伦理问题,探讨嵌入RRI框架来解决人工智能(AI)在产生、应用和发展等过程中滋生的伦理问题的路径,使AI技术更好地造福人类社会。     

管理者从军经历与企业研发投入

自改革开放以来,我国进行了数次大规模的裁军活动,军人管理者已成为中国商界的重要力量。本文基于烙印和高阶理论,以2008-2016年沪深两市A股上市企业为研究对象,实证检验了管理者早期从军经历对企业研发投入的影响及其影响机制。通过面板模型回归,研究发现：第一,有从军经历管理者的企业具有较高的研发投入;第二,管理者从军经历对企业研发投入的影响在创始人从军经历管理者企业、无政治关联企业、非国有企业以及高竞争程度行业中更加显著;第三,管理者风险承担性在管理者从军经历与企业研发投入之间起到了部分中介作用。进一步研究表明,管理者从军经历不仅能够增加企业的研发投入,还能够提高企业的专利技术申请量。本文的研究丰富了管理者异质性领域的相关研究,打开了从军经历管理者对企业研发投入影响的“黑箱”,对上市企业尤其是创新型企业任用管理者提供了指导意义。     

The perceived moral qualities of web sites: implications for persuasion processes in human–computer interaction

This study extended the scope of previous findings in human–computer interaction research within the computers are social actors paradigm by showing that online users attribute perceptions of moral qualities to Websites and, further, that differential perceptions of morality affected the extent of persuasion. In an experiment (N = 138) that manipulated four morality conditions (universalist, relativist, egotistic, control) across worldview, a measured independent variable, users were asked to evaluate a Web site designed to aid them in making ethical decisions. Web sites offered four different types of ethical advice as participants contemplated cases involving ethical quandaries. Perceptions of the Web sites’ moral qualities varied depending on the type of advice given. Further, the Web sites’ perceived morality and participants’ worldview predicted credibility, persuasiveness, and attitudes toward the Web sites.     

Lost in cyberspace: ethical decision making in the online environment

In this study, a 20-item questionnaire was used to elicit undergraduates’ (N = 93) ethical judgment and behavioral intention regarding a number of behaviors involving computers and internet usage. Machiavellianism was found to be uncorrelated with both ethical judgment and behavioral intention. Gender was found to be negatively correlated with both ethical judgment and behavioral intention, such that females judged the behaviors as being less ethical than males, and were less likely to engage in the behaviors than males. A disconnect was found between ethical judgment and behavioral intention, for both males and females, such that the ethical judgment mean for a number of issues was significantly lower (towards the “unethical” end of the continuum) than the behavioral intention mean (towards the “more likely to engage in” end of the continuum). The study raises questions regarding ethical awareness of technology-related issues, and the authors make suggestions for future research.     

Anticipating Ethical Challenges: Is There a Coming Era of Nanotechnology?

In this paper I question the claims made for a ‘coming era of nanotechnology’ and the ethical challenges, it is argued, that are entailed by this particular technological revolution. I argue that such futurist claims are sustained by an untenable modernist narrative which separates the technical and the social. This is exemplified by the work of K. Eric Drexler and his claim that whilst the course of scientific knowledge may remain unpredictable we nevertheless can predict with accuracy the trajectory of technology and particularly the emergence of nanotechnology. The problem then, on the basis of knowledge about the future state of technology, is to make choices now which will forestall unintended and undesirable consequences. Firstly, the paper argues for a radical scepticism towards all forms of forecasting or prediction but especially technological forecasting of the type exemplified in the debate around nanotechnology. Secondly, given this radical scepticism the paper criticises the idea that a prospective ethics can be created on the basis of an assessment of consequences of nanotechnology.     

The “big red button” is too late: an alternative model for the ethical evaluation of AI systems

As a way to address both ominous and ordinary threats of artificial intelligence (AI), researchers have started proposing ways to stop an AI system before it has a chance to escape outside control and cause harm. A so-called “big red button” would enable human operators to interrupt or divert a system while preventing the system from learning that such an intervention is a threat. Though an emergency button for AI seems to make intuitive sense, that approach ultimately concentrates on the point when a system has already “gone rogue” and seeks to obstruct interference. A better approach would be to make ongoing self-evaluation and testing an integral part of a system’s operation, diagnose how the system is in error and to prevent chaos and risk before they start. In this paper, we describe the demands that recent big red button proposals have not addressed, and we offer a preliminary model of an approach that could better meet them. We argue for an ethical core (EC) that consists of a scenario-generation mechanism and a simulation environment that are used to test a system’s decisions in simulated worlds, rather than the real world. This EC would be kept opaque to the system itself: through careful design of memory and the character of the scenario, the system’s algorithms would be prevented from learning about its operation and its function, and ultimately its presence. By monitoring and checking for deviant behavior, we conclude, a continual testing approach will be far more effective, responsive, and vigilant toward a system’s learning and action in the world than an emergency button which one might not get to push in time.     

Is there an ethics of algorithms?

We argue that some algorithms are value-laden, and that two or more persons who accept different value-judgments may have a rational reason to design such algorithms differently. We exemplify our claim by discussing a set of algorithms used in medical image analysis: In these algorithms it is often necessary to set certain thresholds for whether e.g. a cell should count as diseased or not, and the chosen threshold will partly depend on the software designer’s preference between avoiding false positives and false negatives. This preference ultimately depends on a number of value-judgments. In the last section of the paper we discuss some general principles for dealing with ethical issues in algorithm-design.     

Surveillance,Privacy and the Ethics of Vehicle Safety Communication Technologies

Recent advances in wireless technologies have led to the development of intelligent, in-vehicle safety applications designed to share information about the actions of nearby vehicles, potential road hazards, and ultimately predict dangerous scenarios or imminent collisions. These vehicle safety communication (VSC) technologies rely on the creation of autonomous, self-organizing, wireless communication networks connecting vehicles with roadside infrastructure and with each other. As the technical standards and communication protocols for VSC technologies are still being developed, certain ethical implications of these new information technologies emerge: Coupled with the predicted safety benefits of VSC applications is a potential rise in the ability to surveil a driver engaging in her everyday activities on the public roads. This paper will explore how the introduction of VSC technologies might disrupt the “contextual integrity” of personal information flows in the context of highway travel and threaten one’s “privacy in public.” Since VSC technologies and their related protocols and standards are still in the developmental stage, the paper will conclude by revealing how close attention to the ethical implications of the remaining design decisions can inform and guide designers of VSC technologies to create innovate safety applications that increase public safety, but without compromising the value of one’s privacy in public. This work was supported by the National Science Foundation PORTIA Grant No. CNS-0331542, and could not have been completed without the valuable guidance of Prof. Helen Nissenbaum (New York University) and Prof. Dan Boneh (Stanford University). I am grateful to many other colleagues who generously contributed to this work with excellent comments and suggestions, including Emily Clark and Steve Tengler at the VSCC, and Sam Howard-Spink, Joseph Reagle and Tim Weber at New York University. Drafts were further sharpened through opportunities to present at colloquia and conferences sponsored by New York University’s Department of Culture & Communication, the American Association for the Advancement of Science, the Society for Philosophy and Technology, and Computer Ethics: Philosophical Enquiry.     

Anticipating ethical issues in emerging IT

In this essay, a new approach to the ethics of emerging information technology will be presented, called anticipatory technology ethics (ATE). The ethics of emerging technology is the study of ethical issues at the R&D and introduction stage of technology development through anticipation of possible future devices, applications, and social consequences. In the essay, I will first locate emerging technology in the technology development cycle, after which I will consider ethical approaches to emerging technologies, as well as obstacles in developing such approaches. I will argue that any sound approach must centrally include futures studies of technology. I then present ATE and some applications of it to emerging information technologies. In ATE, ethical analysis is performed at three levels, the technology, artifact and application levels, and at each levels distinct types of ethical questions are asked. ATE analyses result in the identification and evaluation of a broad range of ethical issues that can be anticipated in relation to an emerging information technology. This ethical analysis can then be used for ethical recommendations for design or governance.     

An ethical framework in information systems decision making using normative theories of business ethics

As business environments become more complex and reliant on information systems, the decisions made by managers affect a growing number of stakeholders. This paper proposes a framework based on the application of normative theories in business ethics to facilitate the evaluation of IS related ethical dilemmas and arrive at fair and consistent decisions. The framework is applied in the context of an information privacy dilemma to demonstrate the decision making process. The ethical dilemma is analyzed using each one of the three normative theories—the stockholder theory, stakeholder theory, and social contract theory. The challenges associated with the application of these theories are also discussed.     

Maintaining the reversibility of foldings: Making the ethics (politics) of information technology visible

This paper will address the question of the morality of technology. I believe this is an important question for our contemporary society in which technology, especially information technology, is increasingly becoming the default mode of social ordering. I want to suggest that the conventional manner of conceptualising the morality of technology is inadequate – even dangerous. The conventional view of technology is that technology represents technical means to achieve social ends. Thus, the moral problem of technology, from this perspective, is the way in which the given technical means are applied to particular (good or bad) social ends. In opposition to this I want to suggest that the assumed separation, of this approach, between technical means and social ends are inappropriate. It only serves to hide the most important political and ethical dimensions of technology. I want to suggest that the morality of technology is much more embedded and implicit than such a view would suggest. In order to critique this approach I will draw on phenomenology and the more recent work of Bruno Latour. With these intellectual resources in mind I will propose disclosive ethics as a way to make the morality of technology visible. I will give a brief account of this approach and show how it might guide our␣understanding of the ethics and politics of technology by considering two examples of contemporary information technology: search engines and plagiarism detection systems.     

The freedoms of software and its ethical uses

The “free” in “free software” refers to a cluster of four specific freedoms identified by the Free Software Definition. The first freedom, termed “Freedom Zero,” intends to protect the right of the user to deploy software in whatever fashion, towards whatever end, he or she sees fit. But software may be used to achieve ethically questionable ends. This highlights a tension in the provision of software freedoms: while the definition explicitly forbids direct restrictions on users’ freedoms, it does not address other means by which software may indirectly restrict freedoms. In particular, ethically-inflected debate has featured prominently in the discussion of restrictions on digital rights management and privacy-violating code in version 3 of the GPL (GPLv3). The discussion of this proposed language revealed the spectrum of ethical positions and valuations held by members of the free software community. In our analysis, we will provide arguments for upholding Freedom Zero; we embed the problem of possible uses of software in the broader context of the uses of scientific knowledge, and go on to argue that the provision of Freedom Zero mitigates against too great a moral burden—of anticipating possible uses of software—being placed on the programmer and that, most importantly, it facilitates deliberative discourse in the free software community.     

The tragedy of the digital commons

In the paper it is argued that bridging the digital divide may cause a new ethical and social dilemma. Using Hardin's Tragedy of the Commons, we show that an improper opening and enlargement of the digital environment (Infosphere) is likely to produce a Tragedy of the Digital Commons (TDC). In the course of the analysis, we explain why Adar and Huberman's previous use of Hardin's Tragedy to interpret certain recent phenomena in the Infosphere (especially peer-to-peer communication) may not be entirely satisfactory. We then seek to provide an improved version of the TDC that avoids the possible shortcomings of their model. Next, we analyse some problems encountered by the application of classical ethics in the resolution of the TDC. In the conclusion, we outline the kind of work that will be required to develop an ethical approach that may bridge the digital divide but avoid the TDC.     

技术管理过程和评估

技术包含于企业每一个价值活动中,技术变革对任何活动都产生影响,从而影响竞争。因此,建立合理的技术战略和科学的技术管理就变得非常必要。作者认为,技术管理需要考虑把研究与开发(R&D)部门同其它部门的活动相结合;技术管理不仅要考虑企业的技术及相关战略,还要考虑技术战略的可操作性;技术管理需要积极地运用组织中的战略资源,在制订技术战略和具体项目活动时要根据信息流程中的反馈而积极加以调整。     

Unconventional competition – drawing lessons from the military

We explore aspects of unconventional competition between firms. In particular, we address two critical questions. The first is: what happens when firms decide to compete outside the rules and frameworks of conventional competition? In other words, how should firms react to competitors who employ unconventional means of competition that differ from recognised Western ethical and legal norms? Secondly, we examine whether current warfare models provide guidance to firms facing such unconventional competition. To address these two questions, we seek to draw lessons from decision‐making approaches adopted by the military. We come to the conclusion that businesses can learn a great deal from how the military deals with unconventional competition.