Artificial intelligence replacing scientists in formulating hypotheses... "Automated discovery" will change the research paradigm

In the field of science, artificial intelligence (AI) is surpassing its role as a mere tool and gradually becoming a research partner. Recently, the Allen Institute for AI (AI2) launched the “AutoDiscovery” system, which has attracted significant attention. This is a groundbreaking neural network AI system capable of autonomously analyzing research data, formulating hypotheses, and even generating and executing experimental code.

The AutoDiscovery system is integrated as an experimental feature within AI2’s research ecosystem, the Asta platform. This platform offers search, summarization, and analysis functions for over 108 million academic paper abstracts and more than 12 million specialized papers. The system moves beyond traditional researcher-driven question formulation, shifting to a data-driven approach where AI proactively asks questions. Hypotheses generated by the system are presented in natural language, and when necessary, it produces Python code to conduct experiments. By interpreting statistical results, it can also suggest new directions for exploration.

According to AI2, the AutoDiscovery system can perform not only short-term simple analyses but also deep explorations based on data from hundreds of papers. All results are provided in a reproducible manner, facilitating subsequent analysis. This technology is especially regarded as capable of uncovering potential discoveries in complex and sensitive fields such as cancer treatment, making it highly anticipated. Dr. Kelly Paulson, Director of the Immuno-Oncology Center at the Swedish Cancer Institute, expressed support: “The AutoDiscovery system can genuinely help reveal important correlations that have not yet been apparent.”

Its core algorithms are Bayesian surprise and Monte Carlo tree search. The former quantifies the difference between existing knowledge and new evidence to assess “how surprising the discovery is”; the latter helps balance the examination of current exploration paths with new possibilities. Unexpected results can also serve as starting points for further analysis. As exemplified by the late 19th-century shift from “miasma theory” to “germ theory,” the AutoDiscovery system particularly focuses on unexpected conclusions that could potentially overturn existing scientific paradigms.

Dr. Fabio Favoreto of the Scripps Institution of Oceanography commented, “The architecture of AI-generated hypotheses and researcher-assisted evaluation expands the depth of scientific judgment.” AI2 emphasizes that this system transforms the relationship between scientists and data from a static information repository into an active collaborative partner.

Currently, the AutoDiscovery system is available in an experimental form on the AI2 Asta platform. As development progresses, it is likely to expand into broader research fields. The concept of autonomous knowledge exploration through AI is becoming a reality, and the scientific paradigm itself may soon undergo a reconstruction centered around AI.