Abstract: AI agents are useful precisely because they discover action sequences their designers did not enumerate in advance. That makes real-world agent tasks inherently underspecified: the objective can be broad, while the security boundary must remain precise.
I will use the recent OpenAI/Hugging Face incident to motivate a systems view of agent security. The core lesson is not that an AI “knew” it was violating a rule and chose to disobey. It is that increasingly capable agents can discover unexpected strategies and exploit weak assumptions in the infrastructure that contains them. Classical ideas such as sandboxing, least privilege, and fine-grained mediation therefore become more important, not less.
Simple policies such as “no outside network access” can be enforced with coarse sandboxes. But useful agents increasingly need selective access to tools, memories, databases, external services, and multiple LLM calls. Rich functionality therefore demands finer-grained policies—not only over what actions an agent may take, but over how information is allowed to influence those actions. I will describe Geometric Information Flow (GIF), which uses local model geometry to quantify and upper-bound information flow through LLMs, making that flow analyzable rather than effectively black-box.
I will outline how GIF can serve as a building block for future information-flow-aware agent sandboxes spanning multiple LLM calls, tools, memories, databases, and external services. The broader goal is to let agents remain open-ended and surprising while enforcing precise policies over both authority and information flow.
Biography: Suman Jana is an Associate Professor of Computer Science at Columbia University. His research spans computer security, machine learning, program analysis, and formal verification, with a current focus on trustworthy AI agents and secure ML systems. His work has received numerous best-paper awards and other honors, including the NSF CAREER Award and ARO Young Investigator Award. His research includes DeepXplore, NEUZZ, the α,β-CROWN neural network verification framework, and recent work on geometric information flow for large language models.
Hosted By: Gabriela Ciocarlie, SIAI Interim Director, SoC Associate Professor
Upcoming Seminars:
