Abstract: In development of foundation models and LLMs, Euclidean space has been the de facto geometric setting for machine learning architectures. However, at a large scale, real-world data often exhibit inherently non-Euclidean structures, such as multi-way relationships, hierarchies, symmetries, and non-isotropic scaling, in a variety of domains, such as languages, vision, and the natural sciences. It is challenging to effectively capture these structures within the constraints of Euclidean spaces. This talk will cover my research that moves beyond Euclidean geometry to maintain the scaling law for the next generation of foundation models. By adopting these geometries, foundation models could more efficiently leverage the structures. Task-aware adaptability that dynamically reconfigures embeddings to match the geometry of downstream applications, could further enhance efficiency and expressivity. This talk will demonstrate the advantages of hyperbolic geometry in building foundation models from individual non-Euclidean components, LLM pre-training / fine-tuning, retrieval-augmented generation, and mixture of expert models. We further present a comprehensive library that greatly simplifies the process of building and adapting foundation models in non-Euclidean. Finally, the talk will enumerate a few promising directions that are being actively explored in this domain.
Biography: Rex Ying is an assistant professor in the Department of Computer Science at Yale University. His research spans geometric deep learning, multimodal and structured-data foundation models, efficient AI knowledge and memory, trustworthy AI, and AI for science. He develops models that use graphs and geometry to learn expressive, scalable, and interpretable representations, with a growing emphasis on structured memory and reasoning for AI agents and foundation models in non-Euclidean spaces. He has co-developed widely used graph learning methods, including GraphSAGE, PinSAGE, and GNNExplainer, and builds foundation models for applications across science, engineering, medicine, and finance. His honors include the ACM SIGKDD Doctoral Dissertation Award (2022), the Amazon Research Award (2024), the Frontiers of Science Awards (2025 and 2026), and the NSF CAREER Award (2026). His research is supported by the National Science Foundation, ARPA-H, the Betty Irene Moore Foundation, and industry partners including Amazon, Goldman Sachs, Samsung, Snap Research, Google Research, Rokt, General Atomics and NetApp.
Hosted By: Yue Ning, SoC Associate Professor
Upcoming Seminars:
