Haoran (Ryan) Xu 徐浩然
Ph.D. student/The University of Texas at Austin
I am a 4th‑year Ph.D. student at UT Austin, advised by Prof. Amy Zhang, and supported by the Amazon AI PhD Fellowship.
My research asks how to pretrain foundation models with reinforcement learning, and explores the new post-training paradigm that RL pretraining brings. The aim is to keep pretraining stable and scalable to large models, while learning from more diverse data, including sub‑optimal data, to unlock capabilities beyond imitation.
I am open to collaboration; feel free to reach out.
News
- Attending CoRL 2026 in person in Austin.
- Presented my thesis proposal, Towards Scalable RL Pretraining.
- Value Gradient Flow is accepted to ICLR 2026.
- Two papers are accepted to NeurIPS 2025.
- Honored to be selected as an Amazon AI Ph.D. Fellow, with two years of funding.
- Attending RLC 2025 in person in Edmonton.
- Iterative Dual-RL is accepted to ICLR 2025.
- Diffusion-DICE is accepted to NeurIPS 2024.
- Starting my summer internship at MSR New York, working with Alex Lamb and John Langford.
- Attending ICLR 2024 in person in Vienna.
- ODICE is accepted to ICLR 2024 as a spotlight.
- Attending NeurIPS 2023 in person in New Orleans.
- Starting my Ph.D. at UT Austin.
Selected Publications
For a complete list of publications, see Google Scholar.
-
Preprint 2026Horizon-Scalable Q-Learning
-
ICLR 2026Reinforcement Learning via Value Gradient Flow
-
NeurIPS 2025Uni-RL: Unifying Online and Offline RL via Implicit Value Regularization
-
NeurIPS 2024Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning
-
ICLR 2024Spotlight, top 5%ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update
-
ICLR 2023Oral, top 5%Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
-
NeurIPS 2022Oral, top 2%A Policy-Guided Imitation Approach for Offline Reinforcement Learning
-
AAAI 2022Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning
Teaching
-
Spring 2024Reinforcement Learning: Theory and PracticeTeaching Assistant, UT Austin
Service
-
ReviewerICLR, ICML, NeurIPS