Existential risk from artificial ignorance

Existential risk from artificial ignorance is the ongoing threat that unforeseen interactions of control systems could someday result in human extinction. The risks associated with any control systems can be significant, ranging from inconvenience to catastrophic damage. There has been a worldwide debate over Existential risk from artificial general intelligence, but why wait for AI systems to become smarter than humans when artificial ignorance presents risks already? Wherever multiple systems interact, there is an increased chance of errors due to a lack of communication, over-reliance on other systems, or a lack of information about potential risks.

Artificial Ignorance in Art and Literature

The existential risk from artificial ignorance is depicted in the Artificial Ignorance WebComic, as its synopsis indicates: "Artificial Ignorance was a weekly webcomic that follows two robots as they confront their existence, set in both a physical dystopian future without humans and a digital space where their programming is limited to their imagination."

Nithyananda Sangha explained how artificial ignorance plays a significant role in daily life, stating, "Artificial Ignorance has begun to play a leading role."

Current Trends in Artificial Ignorance

The development of "an ignorant (or unaware) iteratively self-improving machine" is occurring on systems lacking ethics, understanding of life, self-awareness, or a global perspective, as detailed in the article on Artificial Ignorance by Steve Moraco. He further notes, "For the first time in human history, there are no longer any technological or conceptual barriers between the current state of the art and a potentially self-designing machine. It's merely a matter of implementation."

The concept of "Trees for the Forest – Ignore and Optimize" (described in Artificial Ignorance - not normal is an opportunity) demonstrates that filtering data for specific characteristics while ignoring the rest is a common and necessary practice. While this is often useful, it carries the risk that users may falsely assume that everything is being checked when, in reality, only a small portion of the data is examined.

In her article AI Ethics: Artificial Intelligence, Robots, and Society, Joanna Bryson wrote, "In fact, AI is here now, and even without AI, our hyperconnected socio-technical culture already creates radically new dynamics and challenges for both human society and our environment."

In an interview with Nick Bostrom, director of the Future of Humanity Institute at Oxford, Ross Andersen questioned, 'In one of your papers on this topic, you note that experts have estimated our total existential risk for this century to be somewhere around 10-20%. I know I can't be alone in thinking that is high. What's driving that?'

Software bots using simple algorithms compete with humans in financial markets and social media. Even within Wikipedia, different bots (whose developers may not know of each other's existence) engage in ongoing battles, increasing volatility and causing significant disruptions due to unintended interactions, as explained in an article by Milena Tsvetkova, Ruth García-Gavilanes, Luciano Floridi, and Taha Yasseri titled "Even good bots fight: The case of Wikipedia". "We have classified high-frequency trading algorithms as malevolent because they exploit markets in ways that increase volatility and precipitate flash crashes. ... Wikipedia is an ecosystem of bots. ... Bots are predictable automatons that do not have the capacity for emotions, meaning-making, creativity, and sociality, and it is hence natural to expect interactions between bots to be relatively predictable and uneventful. ... Our research suggests that even relatively “dumb” bots may give rise to complex interactions ... a system of simple bots may produce complex dynamics and unintended consequences."

"The growing interconnections between people, markets, and networks, together with the development of new technologies, have increased the frequency and impact of large-scale disasters around the globe." "This paper takes a governance perspective by assuming that policy actions should be designed to cope with ignorance and large-scale losses, being the primary features characterizing such emerging catastrophic risks."

Jonathan Yarden discussed the risks of neglecting security, explaining that the growing complexity of computer systems increases the risk level. He wrote, "I'm convinced that the more feature-rich Internet software is, the more bugs it's going to have."

Types of Artificial Ignorance

• Risk that none of the systems was designed to detect.
• Risk that a system was designed to detect, but failed to.
• Safe condition falsely identified as a risk.
• Risk known but each system relies on the other to handle.

Past examples of Artificial Ignorance

A home security system alerted the police to a home, but there was no break-in. The error? The robot vacuum triggered the system. This example illustrates one system's ignorance of others' existence. Would a roomba trigger a home's security system?

False Alarms from NORAD Led to Alert Actions for U.S. Strategic Forces This example shows a system mistakenly identifying a safe condition as a risk. How often have global events, due to this type of artificial ignorance, come close to causing humanity's extinction?

Chernobyl disaster

Catastrophes like the Exxon Valdez oil spill could be prevented with more intelligent control systems. This example demonstrates a lack of awareness of immediate environmental risks.

On December 2, 1984, the Union Carbide pesticide plant in Bhopal, India, leaked methyl isocyanate gas and other poisons into the air. Over half a million people were exposed, leading to over 35,000 deaths. The Bhopal disaster exemplifies over-reliance between systems. The systems designed to prevent gas leaks (mechanical, computerized, and administrative) relied on each other to maintain safe conditions.

References

This article "Existential risk from artificial ignorance" is from Wikipedia. The list of its authors can be seen in its historical. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

📰 Article(s) of the same category(ies)

• Ethics of simulated suffering
• Year 10,000 problem
• Existential risk
• Vulnerable world hypothesis
• Cybergeddon
• Survival Edge Technology
• Center on Long-Term Risk

---