[ad_1]
OpenAI has formed a new team responsible for addressing the potential dangers associated with AI. This team, known as the Catastrophic Risk Team, will prepare to monitor, evaluate, predict and protect against major issues, including nuclear threats. AI models exceed the capabilities of existing models. The team is also led by Aleksander Madry, director of the MIT Center for Deployable Machine Learning and faculty co-lead of the MIT AI Policy Forum.
As advanced generative AI expands and adds new capabilities, it could revolutionize the benefits for all humanity. OpenAI CEO Altman suggested that the government would treat AI as seriously as nuclear weapons. OpenAI also prepares for and strives to prevent catastrophic risks from cutting-edge AI systems. OpenAI is redoubling its efforts to prevent an AI-induced catastrophe through its new team to address potential risks associated with advanced AI.
Catastrophic team
This team works to mitigate chemical, biological and radiological threats, which are said to be acts of AI replicating itself. The preparedness team is also responsible for developing and maintaining a risk-informed development policy that outlines and maintains OpenAI’s research, evaluation, and monitoring of AI models. The modern computer, which determines how military attacks are carried out, is undoubtedly involved, as AI is prone to hallucinations and does not necessarily hold the same philosophies as a human.
The team discussed AI’s ability to deceive people, but also took into account the cyber threats that pose increasingly serious risks. Sam Altman of OpenAI has previously warned about the potential risk of AI, and other leading AI researchers have issued a 22-World Statement. AI could also determine the timing of a nuclear attack, as the groundbreaking AI model exceeds the capabilities currently present in the most advanced existing models.
In addition, to protect against the dangers of groundbreaking AI systems, a plan is being developed to guide the process. This plan ensures transparency, accountability and supervision. The challenge is to recruit new ideas and submissions to prevent catastrophic abuse, offer API credits and potential recruitment opportunities for the best submissions.
OpenAI has also posted job openings for National Security Threats researchers and research engineers, with annual salaries between $200,000 and $370,000. The approach to building AI models includes evaluation and monitoring tools, risk mitigation measures, and a governance structure to oversee the entire model development process. The company also asks participants to imagine the most unique and likely misuses of its model, as these AI models are beyond human capabilities and have the potential to benefit or harm humanity.
OpenAI aims to combat AI risk with current and future AI models
Including cybersecurity, autonomous replication and adaptation, and even extinction-level threats such as chemical, biological, radiological, and nuclear attacks, AI models aim to reduce the risk of AI extinction. This is a global priority alongside other societal-scale risks such as pandemics and nuclear wars.
AI has an incredible impact on the future and poses serious challenges and risks to the way we interact with technology. Having a professional team willing to evaluate upcoming AI models also comes with increasing risks. These focus on three key areas:
- AI can be misused, which could pose a danger to groundbreaking AI systems and those that will emerge in the future.
- If an AI model is stolen, which can be done by a malicious actor,
- Frameworks can be built that monitor, evaluate, predict, and protect against the dangerous capabilities of cutting-edge AI systems.
Additionally, OpenAI has also announced a catastrophic abuse prevention plan. The company is offering $25,000 in API credits for up to ten submissions that publish likely but potentially catastrophic abuses of OpenAI’s AI model. Groundbreaking, beyond-capability AI models aim to study and protect against threats posed by advanced AI capabilities, as they also pose increasingly serious risks.
The teams, which it classifies as competent models with potentially dangerous skills, monitor the company’s AI models to keep them in line with the safety rails and its deployment. There is also a risk that AI models can persuade human users through language and perform autonomous tasks.
The company’s claim to build secure AGI (artificial general intelligence) explains how it plans to track, evaluate, predict, and protect against AI. OpenAI said it takes the full spectrum of security risks related to AI seriously, but did not provide official details or evidence of preparedness efforts. There is also a risk-informed development policy, which includes protective measures and a governance structure to hold AI systems accountable.