OpenAI and Anthropic sign letter to prevent biological weapons developed by AI

The chief executive officers of several major artificial intelligence companies are urging parliamentarians to adopt new laws that make it more difficult for undesirable actors to use their technology for the development of biological weapons.

Google DeepMind Demis Hassabis,OpenAI Sam Altman,Anthropic Dario Amodei and Microsoft Mustafa Suleyman of AI was one of the signatories of an open letter calling for legislation requiring companies selling synthetic DNA and RNA to screen their customers and order to prevent the misuse of genetic material.

This letter, organized by the Unpartisan Progress Institute and the Right-to-Right Innovation Foundation, acknowledges that, given the pace of development of artificial intelligence, “the historical barriers to the knowledge of bad people to acquire biological weapons may indeed be meaningfully weakened”.

Scientist Arthur Kornberg was the first person to successfully synthesize DNA in the 1950s. This process is now automated, with dozens of companies around the world using commercial synthetics to “print” and sell customized gene sequences for scientific research, drug development and diagnosis. Many suppliers sell their products only to qualified researchers, biotechnology companies and educational institutions, but not all of them review their customers or the genetic sequence they order.

In 2017, Canadian researchers used $100,000 worth of mail-order DNA to rebuild the extinct pox virus, which raised alarm. Critics say the same approach can be used to construct smallpox, a closely related lethal virus. Since then, gene synthesis has become cheaper.

Combined with advances in artificial intelligence, it is now feasible to design dangerous new toxins and pathogens using large language models, although some biological training may still be needed to start from scratch to create functional viruses. Although biological terrorist attacks are rare, they have the potential to cause mass casualties, public panic and economic loss. A major concern is that artificially intelligent pathogens may cause a global pandemic, whether intentional or unintentional.

The microbial scientist and biosafety expert at Stanford University, who signed the letter, David Relman, stated: “Agent intelligence tools enable users to identify very quickly where to look for unscreened sequences.” “If properly informed, they can also tell you how to change the nature of the order so that even those who are being screened may not be able to detect what you want to do.”

The signatories include other scientists, national security experts and executives of Twist Bioscience and Ansa Biotechnologies, a genetic synthesis company. These companies are members of the International Union of Genetic Synthesis, which was established in 2009 to implement voluntary screening practices. Many companies have used software to screen “sequence of concern” orders that may lead to the toxicity or pathogenicity of organisms.

The vice president of Twist Bioscience Policy and Biosafety, James Diggans, said, "If you have the technology to synthesize DNA, then you should ensure that it is used responsibly, in part to ensure that you know what you're making and who you're making for." The company has supported the implementation of formal rules for many years.

Federal guidelines introduced during the Biden Government require federally funded scientists and companies to order synthetic gene sequences from the vendor that screens for procurement. A bipartisan bill introduced in the Senate earlier this year will require all gynthesizing providers operating in the United States to screen for the presence of undesirable actors or dangerous pathogens among their clients.

But the screening tool is not perfect. Last year, a study published by Microsoft researchers showed that artificially intelligent protein design tools can generate potentially dangerous genetic sequences that can be screened by companies. These models present new protein sequences similar to known hazardous protein structures.

The former president of Y Combinator and partner of Safe AI Fund Geoff Ralston believes that artificial intelligence laboratories with biological models should screen users themselves.

“It would be very difficult, if not impossible, to ask a model to help you do something dangerous that is imminent”, Larston said, and he signed the letter.

Lehrman agreed that the provisions on screening procedures were only part of the solution. “We must have other control points, given that in some cases screening may fail”, he said. "This is where artificial intelligence companies have to stand."