As artificial intelligence becomes central to scientific discovery, researchers face a growing but often overlooked risk: the AI models, datasets, and automated systems they depend on can be compromised in ways that conventional cybersecurity tools are not designed to detect.
A new project called VERITAS (VERified Infrastructure for Trustworthy AI in Science), led by principal investigator Anita Nikolich, research scientist and director of research and technology innovation, will address this gap by establishing AI Assurance as a core function of scientific research infrastructure. Funded through a three-year, $896,000 grant from the National Science Foundation's Cybersecurity Innovation for Cyberinfrastructure program, VERITAS brings together experts in adversarial AI, research cyberinfrastructure, data science, and workforce development. The project aims to develop practical methods for documenting, reviewing, and stress-testing AI systems before they are used in high-impact scientific workflows.
A blind spot in how science secures AI
Traditional cybersecurity focuses on preventing unauthorized access, catching malware, and stopping data theft. AI-enabled research introduces additional risks that may not trigger conventional security alerts.
A poisoned dataset, for example, may appear statistically normal while causing a model to produce unreliable results. A backdoored model downloaded from a public repository may contain no recognizable malware and may operate normally until a particular input activates its hidden behavior. An autonomous AI agent may have excessive permissions that allow it to alter data, invoke laboratory tools, or manipulate a research workflow.
In each case, the infrastructure may appear secure while the scientific result is compromised.
"We cannot simply bolt traditional cybersecurity onto AI-driven science," said Nikolich. "When a poisoned dataset or backdoored model produces an answer that looks plausible but is subtly wrong, no firewall or virus scanner is likely to catch it. The researchers doing our most important scientific work deserve assurance that the AI systems they rely on are documented, tested, and behaving as intended."
According to Nikolich, rather than requiring scientists to become cybersecurity experts or expecting cybersecurity teams to become machine-learning specialists, VERITAS will integrate AI Assurance into the research infrastructure scientists already use. The project has three connected components:
Model and data documentation. VERITAS will pilot standardized model cards and dataset datasheets for large scientific computing allocations. Similar to nutrition labels on packaged food, these documents describe where a model or dataset came from, how it was created or modified, its intended use, its known limitations, and the assumptions researchers should understand before reusing it. The goal is to improve transparency, reproducibility, and the ability to trace problems through complex AI workflows.
Operational AI security services. VERITAS will pilot a new AI Assurance Engineer role at the National Center for Supercomputing Applications (NCSA). The engineer will review selected technically novel AI projects before deployment, scan model files for unsafe or malicious behavior, examine software for vulnerabilities, and assess the risks around uses of autonomous agents.
Model and data integrity challenges. Through the National Data Platform (NDP) Education Hub, VERITAS will create hands-on challenges that train students to detect poisoned data, inspect potentially compromised models, evaluate agent permissions, and identify weaknesses in scientific AI workflows.
Finding vulnerabilities before they become scientific failures
AI red-teaming—deliberately attacking an AI system to find its weaknesses before an adversary does—is now a well-established field. It has rarely been brought into scientific research, where a manipulated model produces a false result that can pass for legitimate science. VERITAS is among the first efforts to adapt the practice to scientific cyberinfrastructure.
"AI systems can fail in ways that are difficult to distinguish from legitimate scientific results," said Nikolich. "Proactive red teaming allows us to identify those weaknesses before a vulnerable model or agent becomes embedded in a research pipeline. The objective is to help research teams make their systems more trustworthy and resilient."
Building the AI Assurance workforce
VERITAS will also help prepare students for careers at the intersection of machine learning, cybersecurity, and scientific computing. Participants in the project's challenges will work with realistic scientific models, datasets, and infrastructure using NDP while learning about responsible disclosure practices.
By embedding documentation, security review, adversarial assessment, and workforce development into existing scientific cyberinfrastructure, VERITAS seeks to create a model for AI Assurance that can be adopted by supercomputing centers, research institutions, and national-scale AI infrastructure providers.
"AI is now part of the scientific workflow," Nikolich said. "We need to protect its integrity just as seriously as we protect the networks and computing systems around it."