Together, those gaps mean an AI can confidently recommend actions that look effective in training and testing but perform poorly once deployed or fail to carry over when the surrounding environment changes.

Electrical Engineering and Computer Science Professor Junzhe Zhang has received to develop artificial intelligence systems that make more dependable decisions when the data they learn from is incomplete or no longer matches real-world conditions.

The project aims to develop learning methods that account for both problems so that AI decision-making becomes more efficient, robust, generalizable and easier to interpret.

The work centers on reinforcement learning, a widely used approach to teaching AI systems to make a series of decisions. Its practical use is limited because it typically assumes that no hidden factors influenced how the data were collected and that the environment stays the same between training and deployment.

Those assumptions rarely hold in real-world data.

“In fields like medicine or robotics, a bad decision has real consequences. This grant will help develop AI that not only performs well in the lab but also holds up when it meets the messy real world,” says Zhang.

Zhang’s team will develop a framework called Approximate Causal Reinforcement Learning. It connects the formal requirements of causal inference theory with the limited knowledge about a system that reinforcement learning practitioners usually have. The real environment is modeled as a structural causal model, while the AI learner works from a simplified version that captures both hidden influences on past decisions and differences between training and deployment settings.

The project has three goals:

• Develop methods that use hidden-factor-affected historical data to calculate reliable performance bounds for candidate strategies and use those bounds to improve strategies safely.
• Create algorithms that learn what experts are trying to achieve by watching their demonstrations, even when the experts relied on information the AI cannot see, along with techniques to help learning when feedback is sparse.
• Build methods that let AI agents adapt to changing environments and learn faster by first training on systematically simplified versions of the target task.