Hello! I am Haoran Xu (徐桩焢 in Chinese). I am a first-year Ph.D. student at UT Austin, advised by Prof. Amy Zhang. Previously, I worked at Institute for AI Industry Research (AIR), Tsinghua University, Microsoft Research Asia and JD.com on data-driven Reinforcement Learning and its applications. I obtained Bachelor & Master's degree from Xidian University, advised by Xianyuan Zhan and Zheng Yu.

My ultimate research dream is to maximally leveraging prior information to facilitate the development of machine autonomy. Towards this goal, my work primarily focused on offline RL, imitation learning, and more recently, generative model with RL and RLHF. I am open to collaboration, feel free to reach me out!

Some links: Github / Twitter / Google Scholar / haoran.xu@utexas.edu

News

  • One paper on offline RL is accepted to ICLR 2024 as spotlight.
  • πŸ‡ΊπŸ‡Έ I am attending NeurIPS 2023 in-person at New Orleans.
  • One paper on offline multi-agent RL is accepted to NeurIPS 2023.
  • πŸŽ“ Starting my PhD at UT Austin.
  • πŸ‡·πŸ‡Ό I am attending ICLR 2023 in-person at Kigali.
  • 🌟 Three papers I like very much on offline RL and reward learning are accepted to ICLR 2023, including one oral.
  • Honored to be selected as Top Reviewers in NeurIPS 2022.
  • One paper on offline RL is accepted to NeurIPS 2022.
  • One paper on offline IL is accepted to ICML 2022.

Publications (* marks equal contribution)

  • ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update Liyuan Mao*, Haoran Xu*, Weinan Zhang, Xianyuan Zhan ICLR 2024 (Spotlight, Top 5%) 2024 Paper | Code |
  • Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, Xianyuan Zhan ICLR 2023 (Oral, Top 5%) 2023 Paper | Code | Slide
  • A Policy-Guided Imitation Approach for Offline Reinforcement Learning Haoran Xu*, Li Jiang*, Jianxiong Li, Xianyuan Zhan NeurIPS 2022 (Oral, Top 2%) 2023 Paper | Code | Thread | Media
  • Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations Haoran Xu, Xianyuan Zhan, Honglei Yin, Huiling Qin ICML 2022 2022 Paper | Code | Thread
  • Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning Haoran Xu, Xianyuan Zhan, Xiangyu Zhu AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code | Slides
  • DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning Xianyuan Zhan*, Haoran Xu*, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code
  • PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang Preprint 2024 Paper | Code
  • ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update Liyuan Mao*, Haoran Xu*, Weinan Zhang, Xianyuan Zhan ICLR 2024 (Spotlight, Top 5%) 2024 Paper |
  • Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization Xiangsen Wang, Haoran Xu, Yinan Zheng, Xianyuan Zhan NeurIPS 2023 2023 Paper | Code
  • SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning Qin Zhang*, Linrui Zhang*, Haoran Xu, Li Shen, Bowen Wang, Xueqian Wang, Bo Yuan, Yongzhe Chang, Dacheng Tao ICLR 2023 SR4AD Workshop 2023 Paper
  • Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, Xianyuan Zhan ICLR 2023 (Notable Top 5%) 2023 Paper | Code
  • Mind the Gap: Offline Policy Optimizaiton for Imperfect Rewards Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Qing-Shan Jia, Ya-Qin Zhang ICLR 2023 2023 Paper | Code
  • When data geometry meets deep function: Generalizing offline reinforcement learning Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, Ya-Qin Zhang ICLR 2023 2023 Paper | Code
  • A Policy-Guided Imitation Approach for Offline Reinforcement Learning Haoran Xu*, Li Jiang*, Jianxiong Li, Xianyuan Zhan NeurIPS 2022 (Oral, Top 2%) 2022 Paper | Code | Slides | Media
  • Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations Haoran Xu, Xianyuan Zhan, Honglei Yin, Huiling Qin ICML 2022 2022 Paper | Code | Slides
  • Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning Haoran Xu, Xianyuan Zhan, Xiangyu Zhu AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code | Slides
  • DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning Xianyuan Zhan*, Haoran Xu*, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng AAAI 2022 (Spotlight @ ICML 2021 RL4RealLife workshop) 2022 Paper | Code
  • Discriminator-Guided Model-Based Offline Imitation Learning Wenjia Zhang, Haoran Xu, Haoyi Niu, Peng Cheng, Ming Li, Heming Zhang, Guyue Zhou, Xianyuan Zhan CoRL 2022 2022 Paper
  • Model-Based Offline Planning with Trajectory Pruning Xianyuan Zhan, Xiangyu Zhu, Haoran Xu IJCAI 2022 2022 Paper | Code
  • ECoalVis: Visual Analysis of Control Strategies in Coal-fired Power Plants Shuhan Liu, Di Weng, Yuan Tian, Zikun Deng, Haoran Xu, Xiangyu Zhu, Honglei Yin, Xianyuan Zhan, Yingcai Wu IEEE VIS 2022 2022 Paper | Code
  • Multi-Memory enhanced Separation Network for Indoor Temperature Prediction Zhewen Duan, Xiuwen Yi, Peng Li, Dekang Qi, Yexin Li, Haoran Xu, Yanyong Huang, Junbo Zhang, Yu Zheng DASFAA 2022 2022 Paper
  • Offline Reinforcement Learning with Soft Behavioral Regularization Haoran Xu, Xianyuan Zhan, Jianxiong Li, Honglei Yin NeurIPS 2021 Offline RL Workshop 2021 Paper | Code
  • Robust Spatio-Temporal Purchase Prediction via Deep Meta Learning Huiling Qin, Songyu Ke, Xiaodu Yang, Haoran Xu, Xianyuan Zhan, Yu Zheng AAAI 2021 2021 Paper

Professional Services

Reviewer for ICLR, ICML, NeurIPS