Center for AI Safety
 | |
| Formation | 2022 |
|---|---|
| Headquarters | San Francisco, California |
Director | Dan Hendrycks |
| Website | https://www.safe.ai/ |
The Center for AI Safety (CAIS) is a San Francisco–based nonprofit organization that promotes the safe development and deployment of artificial intelligence (AI). CAIS's work encompasses research in technical AI safety and AI ethics, as well as field building to support research in AI safety.[1][2]
In May 2023, CAIS published a statement on AI risk of extinction signed by hundreds of professors of AI, leaders of major AI companies, and other public figures.[3][4][5][6][7]
Research
CAIS's AI alignment research focuses on training models to represent and safely optimize human values. To facilitate the development of agents that avoid causing harm, researchers from CAIS introduced an environment suite to robustly evaluate whether AI agents can be steered with an "artificial conscience" to act ethically.[8][9]
In 2023, CAIS published research examining how natural selection and competitive pressures could shape the goals of artificial agents.[10][9] It also published "An Overview of Catastrophic AI Risks", which details risk scenarios and risk mitigation strategies. Risks described include the use of AI in autonomous warfare or for engineering pandemics, as well as AI capabilities for deception and hacking.[11][9] This followed another paper outlining "speculative" risks from AI such as "eroded epistemics" and "proxy gaming", a term for AIs optimizing objectives in undesirable ways.[12] Another work, conducted in collaboration with researchers at Carnegie Mellon University, described an automated way to discover adversarial attacks of large language models that bypass safety measures, highlighting the inadequacy of current safety systems.[13][14]
Activities
In February 2022, CAIS helped provide recommendations for the US National Institute of Standards and Technology (NIST) to promote AI safety.[15][16]
In December 2022, CAIS helped organize a workshop on machine learning safety at the Conference on Neural Information Processing Systems (NeurIPS). This included a call for papers focusing on robustness, monitoring, alignment, and systemic safety.[17][18]
Other initiatives include a compute cluster to support AI safety research, an online course titled "Intro to ML Safety", and a fellowship for philosophy professors to address conceptual problems.[9]
See also
References
- ↑ "AI poses risk of extinction, tech leaders warn in open letter. Here's why alarm is spreading". USA TODAY. 31 May 2023.
- ↑ "Our Mission | CAIS". www.safe.ai. Retrieved 2023-04-13.
- ↑ Center for AI Safety's Hendrycks on AI Risks, Bloomberg Technology, 31 May 2023
- ↑ Roose, Kevin (2023-05-30). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York Times. ISSN 0362-4331. Retrieved 2023-06-03.
- ↑ "Artificial intelligence warning over human extinction – all you need to know". The Independent. 2023-05-31. Retrieved 2023-06-03.
- ↑ Lomas, Natasha (2023-05-30). "OpenAI's Altman and other AI giants back warning of advanced AI as 'extinction' risk". TechCrunch. Retrieved 2023-06-03.
- ↑ Castleman, Terry (2023-05-31). "Prominent AI leaders warn of 'risk of extinction' from new technology". Los Angeles Times. Retrieved 2023-06-03.
- ↑ "What Would Jiminy Cricket Do? Towards Agents That Behave Morally". neurips.cc. Retrieved 2023-07-17.
- ↑ 9.0 9.1 9.2 9.3 Scharfenberg, David (July 6, 2023). "Dan Hendrycks from the Center for AI Safety hopes he can prevent a catastrophe". The Boston Globe. Retrieved 2023-07-09.
- ↑ Hendrycks, Dan (2023-03-28). "Natural Selection Favors AIs over Humans". arXiv:2303.16200 [cs.CY].
- ↑ Hendrycks, Dan; Mazeika, Mantas; Woodside, Thomas (2023). "An Overview of Catastrophic AI Risks". arXiv:2306.12001.
- ↑ Gendron, Will. "An AI safety expert outlined a range of speculative doomsday scenarios, from weaponization to power-seeking behavior". Business Insider. Retrieved 2023-07-17.
- ↑ Metz, Cade (2023-07-27). "Researchers Poke Holes in Safety Controls of ChatGPT and Other Chatbots". The New York Times. Retrieved 2023-07-27.
- ↑ "Universal and Transferable Attacks on Aligned Language Models". llm-attacks.org. Retrieved 2023-07-27.
- ↑ "Nvidia moves into A.I. services and ChatGPT can now use your credit card". Fortune. Retrieved 2023-04-13.
- ↑ "Request for Information to the Update of the National Artificial Intelligence Research and Development Strategic Plan: Responses" (PDF). National Artificial Intelligence Initiative. March 2022.
- ↑ "NeurIPS 2022". neurips.cc. Retrieved 2023-04-13.
- ↑ "2022 NeurIPS ML Safety Workshop". neurips2022.mlsafety.org. Retrieved 2023-04-13.
This article "Center for AI Safety" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Center for AI Safety. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
|
This page exists already on Wikipedia. |
