OpenAI’s technology and revenue is ahead of Anthropic’s, but when it comes to practices aimed at making sure people don’t use generative artificial intelligence to harm society, OpenAI seems to be playing catch-up. In a 26-page document published Monday, OpenAI discussed how it evaluates AI models for “catastrophic risks” before selling them to the public.
The move comes three months after Anthropic published its own, 22-page document on how it’s getting ahead of its AI’s dangers. The firm’s co-founder, Sam McCandlish, is in charge of the policies, according to someone with direct knowledge. Given that Anthropic’s leaders left OpenAI in 2020 over safety disagreements (more on that here), it’s not a surprise that Anthropic got a head start in this field.
Until seven months ago, OpenAI didn’t have someone in charge of rigorously quantifying the risks of upcoming models the way Anthropic does. That’s when OpenAI hired Aleksander Madry, a professor at the Massachusetts Institute of Technology, to lead its “preparedness” team. Madry and his crew come up with ways to measure and predict how upcoming models could be used for ill purposes, such as to breach cybersecurity defenses, develop weapons, or persuade people to “change their beliefs.” Another issue the team wants to guard against: an “intelligence explosion,” meaning models that can “survive and replicate in the wild,” evading human controls and “creating a runaway process of self-improvement.”
0 Commentaires