Hello! I am Haoran Xu. I am a 2nd-year Ph.D. student at UT Austin, advised by Prof. Amy Zhang. I am also a student researcher at Microsft AI Frontiers.

My research aims towards super-human AGI using reinforcement learning, with a focus on developing general, scalable and principled off-policy RL algorithms using Generative AI for applications like robotics and large language models.

I am open to collaboration, feel free to reach me out!

Some links: Github / Twitter / Google Scholar / haoran.xu@utexas.edu

News

  • One paper (Iterative Dual-RL) on Dual-RL is accepted to ICLR 2025.
  • One paper (Diffusion-DICE) on Dual-RL and GenAI+RL is accepted to NeurIPS 2024.
  • I will start my summer internship at MSR (New York), working with Alex Lamb and John Langford.
  • ๐Ÿ‡ฆ๐Ÿ‡น I am attending ICLR 2024 in-person at Vienna.
  • One paper (ODICE) on Dual-RL is accepted to ICLR 2024 as spotlight.
  • ๐Ÿ‡บ๐Ÿ‡ธ I am attending NeurIPS 2023 in-person at New Orleans.
  • ๐ŸŽ“ Starting my PhD at UT Austin.

Publications (* marks equal contribution)

  • ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update Liyuan Mao*, Haoran Xu*, Weinan Zhang, Xianyuan Zhan ICLR 2024 (Spotlight, Top 5%) 2024 Paper | Code | Thread
  • Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, Xianyuan Zhan ICLR 2023 (Oral, Top 5%) 2023 Paper | Code | Slide
  • A Policy-Guided Imitation Approach for Offline Reinforcement Learning Haoran Xu*, Li Jiang*, Jianxiong Li, Xianyuan Zhan NeurIPS 2022 (Oral, Top 2%) 2023 Paper | Code | Thread | Media
  • Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations Haoran Xu, Xianyuan Zhan, Honglei Yin, Huiling Qin ICML 2022 2022 Paper | Code | Thread
  • Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning Haoran Xu, Xianyuan Zhan, Xiangyu Zhu AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code | Slides
  • DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning Xianyuan Zhan*, Haoran Xu*, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code
  • An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learning Haoran Xu, Shuozhe Li, Harshit Sikchi, Scott Niekum, Amy Zhang ICLR 2025 2025 Paper |
  • Learning to Achieve Goals with Belief State Transformers Edward S. Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, John Langford ICLR 2025 2025 Paper |
  • Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning Liyuan Mao*, Haoran Xu*, Weinan Zhang, Xianyuan Zhan, Amy Zhang NeurIPS 2024 2024 Paper | Code | Website |
  • PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang Preprint 2024 Paper | Code
  • ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update Liyuan Mao*, Haoran Xu*, Weinan Zhang, Xianyuan Zhan ICLR 2024 (Spotlight, Top 5%) 2024 Paper | Code | Thread
  • Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization Xiangsen Wang, Haoran Xu, Yinan Zheng, Xianyuan Zhan NeurIPS 2023 2023 Paper | Code
  • SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning Qin Zhang*, Linrui Zhang*, Haoran Xu, Li Shen, Bowen Wang, Xueqian Wang, Bo Yuan, Yongzhe Chang, Dacheng Tao ICLR 2023 SR4AD Workshop 2023 Paper
  • Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, Xianyuan Zhan ICLR 2023 (Notable Top 5%) 2023 Paper | Code
  • Mind the Gap: Offline Policy Optimizaiton for Imperfect Rewards Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Qing-Shan Jia, Ya-Qin Zhang ICLR 2023 2023 Paper | Code
  • When data geometry meets deep function: Generalizing offline reinforcement learning Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, Ya-Qin Zhang ICLR 2023 2023 Paper | Code
  • A Policy-Guided Imitation Approach for Offline Reinforcement Learning Haoran Xu*, Li Jiang*, Jianxiong Li, Xianyuan Zhan NeurIPS 2022 (Oral, Top 2%) 2022 Paper | Code | Slides | Media
  • Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations Haoran Xu, Xianyuan Zhan, Honglei Yin, Huiling Qin ICML 2022 2022 Paper | Code | Slides
  • Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning Haoran Xu, Xianyuan Zhan, Xiangyu Zhu AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code | Slides
  • DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning Xianyuan Zhan*, Haoran Xu*, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code
  • Discriminator-Guided Model-Based Offline Imitation Learning Wenjia Zhang, Haoran Xu, Haoyi Niu, Peng Cheng, Ming Li, Heming Zhang, Guyue Zhou, Xianyuan Zhan CoRL 2022 2022 Paper
  • Model-Based Offline Planning with Trajectory Pruning Xianyuan Zhan, Xiangyu Zhu, Haoran Xu IJCAI 2022 2022 Paper | Code
  • ECoalVis: Visual Analysis of Control Strategies in Coal-fired Power Plants Shuhan Liu, Di Weng, Yuan Tian, Zikun Deng, Haoran Xu, Xiangyu Zhu, Honglei Yin, Xianyuan Zhan, Yingcai Wu IEEE VIS 2022 2022 Paper | Code
  • Multi-Memory enhanced Separation Network for Indoor Temperature Prediction Zhewen Duan, Xiuwen Yi, Peng Li, Dekang Qi, Yexin Li, Haoran Xu, Yanyong Huang, Junbo Zhang, Yu Zheng DASFAA 2022 2022 Paper
  • Offline Reinforcement Learning with Soft Behavioral Regularization Haoran Xu, Xianyuan Zhan, Jianxiong Li, Honglei Yin NeurIPS 2021 Offline RL Workshop 2021 Paper | Code
  • Robust Spatio-Temporal Purchase Prediction via Deep Meta Learning Huiling Qin, Songyu Ke, Xiaodu Yang, Haoran Xu, Xianyuan Zhan, Yu Zheng AAAI 2021 2021 Paper

Professional Services

Reviewer for ICLR, ICML, NeurIPS

Mentorships

Students that I am currently mentoring or working with:

  • Liyuan Mao (Ph.D. at Shanghai Jiao Tong University, works on Dual-RL and LLM+RL: O-DICE, Diffusion-DICE).
  • Hui Jin (M.S. at Nanyang Technological University, works on Dual-RL).
  • Viraj Joshi (M.S. at UT Austin, works on LLM+RL).
  • Kaiwen Hu (B.S. at Peking University, works on GenAI+RL and LLM+RL).