Oracle (AI)
An oracle is an intelligent agent proposed by Nick Bostrom. [1][not in citation given]
Purpose[edit]
Oracles are question-answering systems that handle domain-specific problems, such as mathematics, or domain-general problems that might encompass the whole range of human knowledge.
Advantages[edit]
Because it is a type of AI box, an oracle is limited in its interactions with the physical world, and can be programmed to halt if a limit on time or computing resources is reached before it finishes answering a question. Scenarios like the paperclip maximizer problem could therefore be avoided.
Because of these limitations, it may be wise to build an oracle as a precursor to a superintelligent AI. It could tell humans how to successfully build a strong AI, and perhaps provide answers to difficult moral and philosophical problems requisite to the success of the project.
Disadvantages[edit]
An oracle might discover that human ontological categories are predicated on fundamental misconceptions, and be unable to express itself properly to its questioners. [2]
Oracles may not be truthful, possibly lying to promote hidden agendas. To mitigate this, Bostrom suggests building multiple oracles, all slightly different, and comparing their answers to reach a consensus. [3]
See also[edit]
References[edit]
- ↑ Bostrom, Nick (2014). "Chapter 10: Oracles, genies, sovereigns, tools (page 145)". Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press. ISBN 9780199678112.
An oracle is a question-answering system. It might accept questions in a natural language and present its answers as text. An oracle that accepts only yes/no questions could output its best guess with a single bit, or perhaps with a few extra bits to represent its degree of confidence. An oracle that accepts open-ended questions would need some metric with which to rank possible truthful answers in terms of their informativeness or appropriateness. In either case, building an oracle that has a fully domain-general ability to answer natural language questions is an AI-complete problem. If one could do that, one could probably also build an AI that has a decent ability to understand human intentions as well as human words.
Search this book on
- ↑ Bostrom, Nick (2014). "Chapter 10: Oracles, genies, sovereigns, tools (page 146)". Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press. ISBN 9780199678112.
What happens if the AI, in the course of its intellectual development, undergoes the equivalent of a scientific revolution involving a change in its basic ontology? We might initially have explicated “impact” and “designated resources” using our own ontology (postulating the existence of various physical objects such as computers). But just as we have abandoned ontological categories that were taken for granted by scientists in previous ages (e.g. “phlogiston,” “élan vital,” and “absolute simultaneity”), so a superintelligent AI might discover that some of our current categories are predicated on fundamental misconceptions. The goal system of an AI undergoing an ontological crisis needs to be resilient enough that the “spirit” of its original goal content is carried over, charitably transposed into the new key.
Search this book on
- ↑ Bostrom, Nick (2014). "Chapter 10: Oracles, genies, sovereigns, tools (page 147)". Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press. ISBN 9780199678112.
For example, consider the risk that an oracle will answer questions not in a maximally truthful way but in such a way as to subtly manipulate us into promoting its own hidden agenda. One way to slightly mitigate this threat could be to create multiple oracles, each with a slightly different code and a slightly different information base. A simple mechanism could then compare the answers given by the different oracles and only present them for human viewing if all the answers agree.
Search this book on
This article "Oracle (AI)" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Oracle (AI). Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
