Introduction
Prof. Xiangyang Ji 季向阳
Department of Automation, Tsinghua University
Room 825, Main Building, Beijing 100084
xyji@tsinghua.edu.cn · Faculty Homepage
The Intelligent Decision-Making Team (THU-IDM) at Tsinghua University is directed by Professor Xiangyang Ji. Our research is centered on advancing decision-making intelligence, with a particular emphasis on generative modeling, reinforcement learning, and foundation models. The team comprises members with diverse interdisciplinary expertise spanning automation, mathematics, physics, and computer science, and maintains strong collaborative ties with international scholars.
Reinforcement Learning
Offline RL
Meta-RL
MARL
LLM for RL
Robust Adaptation
Reasoning Models
Multi-Task Learning
Main Building, Tsinghua University
News
Sep 18, 2025
Code for paper Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? is now publicly available! Visit the GitHub repository to access it!
Jul 9, 2025
Our paper Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? is made public! First trial to successfully predict the prompt difficulty online for RL finetuning of reasoning models, which can significantly accelerate the RL finetuning process. Check out the full paper for more details!
May 1, 2025
Our paper Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments has been accepted to ICML2025 with high scores (5443)! This work introduces PDTS to accommodate fast and robust sequential decision-making, unlocking the power of robust active task sampling.
Mar 26, 2025
THU-IDM team website was updated!
Jan 19, 2025
Attention! Our paper Model Predictive Task Sampling for Efficient and Robust Adaptation is made public! First trial to successfully predict the optimization outcome after zero/few-shot adaptation given some tasks and model predictive task sampling (MPTS) significantly improves the adaptation robustness of multimodal foundation models and sequential decision-making with learning efficiency retained!
Dec 10, 2024
Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning was accepted to AAAI2025. Congratulations to Yun Qu and Yuhang!
Nov 17, 2024
Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation got accepted to KDD2025 Research Track, the literature work to generate task distributions for meta learning!
Sep 26, 2024
Doubly Mild Generalization for Offline Reinforcement Learning and Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression got accepted to NeurIPS2024. Congratulations to Yixiu!
Sep 26, 2024
The multi-task learning SOTA work GO4Align: Group Optimization for Multi-Task Alignment got accepted to NeurIPS2024. Congratulations to Jiayi!
May 1, 2024
LLM-Empowered State Representation for Reinforcement Learning was accepted to ICML2024. Congratulations to Boyuan and Yun Qu!
That's all news!
No news matching current filter.
Publications
arXiv 2026
Reasoning Models
Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models
Yun Qu, Qi Wang, Yixiu Mao, Heming Zou, Yuhang Jiang, Weijie Liu, Clive Bai, Kai Yang, Yangkun Chen, Saiyong Yang, Xiangyang Ji
ICLR 2025
DynaPrompt: Dynamic Test-Time Prompt Tuning
Zehao Xiao, Shilin Yan, Jack Hong, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiayi Shen, Qi Wang, Cees G. M. Snoek
KDD 2025
Robust Adaptation
Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation
Cheems Wang, Yiqin Lv, Yixiu Mao, Yun Qu, Yi Xu, Xiangyang Ji
arXiv 2025
SFT
Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji
arXiv 2025
Offline RL
Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning
Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji
arXiv 2025
Multi-Task Learning
VAO: Validation-Aligned Optimization for Cross-Task Generative Auto-Bidding
Yiqin Lv, Zhiyu Mou, Miao Xu, Jinghao Chen, Qi Cheems Wang, Yixiu Mao, Yun Qu, Rongquan Bai, Chuan Yu, Jian Xu, Bo Zheng, Xiangyang Ji
NeurIPS 2024
Offline RL
Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression
Yixiu Mao, Qi Wang, Chen Chen, Yun Qu, Xiangyang Ji
NeurIPS 2024
Robust Adaptation
Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning
Yiqin Lv, Qi Wang, Dong Liang, Zheng Xie
arXiv 2024
LLM4RL
Choices are More Important than Efforts: LLM Enables Efficient Multi-Agent Exploration
Yun Qu, Boyuan Wang, Yuhang Jiang, Jianzhun Shao, Yixiu Mao, Cheems Wang, Chang Liu, Xiangyang Ji
AAAI 2023
Offline RL
DARL: distance-aware uncertainty estimation for offline reinforcement learning
Hongchang Zhang, Jianzhun Shao, Shuncheng He, Yuhang Jiang, Xiangyang Ji
ICLR 2023
Offline RL
In-sample Actor Critic for Offline Reinforcement Learning
Hongchang Zhang, Yixiu Mao, Boyuan Wang, Shuncheng He, Yi Xu, Xiangyang Ji
ICLR 2023
Offline RL
Pessimistic Policy Iteration for Offline Reinforcement Learning
Hongchang Zhang, Boyuan Wang, Yixiu Mao, Jianzhun Shao, Yuhang Jiang, Yi Xu, Xiangyang Ji
NeurIPS 2023
Multi-Task Learning
Episodic Multi-Task Learning with Heterogeneous Neural Processes
Jiayi Shen, Xiantong Zhen, Qi Wang, Marcel Worring
NeurIPS 2023
Robust Adaptation
A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Qi Wang, Yiqin Lv, Yanghe Feng, Zheng Xie, Jincai Huang
NeurIPS 2023
Offline RL
Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks
Yun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Lin Liu, Junfeng Yang, Lin Lai, Hongyang Qin, Minwen Deng, Juchao Zhuo, Deheng Ye, Qiang Fu, Wei Yang, Guang Yang, Lanxiao Huang, Xiangyang Ji
The Innovation 2023
Large-scale generative simulation artificial intelligence: The next hotspot
Qi Wang, Yanghe Feng, Jincai Huang, Yiqin Lv, Zheng Xie, Xiaoshan Gao
AAAI 2022
Offline RL
State deviation correction for offline reinforcement learning
Hongchang Zhang, Jianzhun Shao, Yuhang Jiang, Shuncheng He, Guanwen Zhang, Xiangyang Ji
AAAI 2022
Unsupervised RL
Wasserstein unsupervised reinforcement learning
Shuncheng He, Yuhang Jiang, Hongchang Zhang, Jianzhun Shao, Xiangyang Ji
COLT 2022
RL Theory
Horizon-free reinforcement learning in polynomial time: the power of stationary policies
Zihan Zhang, Xiangyang Ji, Simon Du
ICML 2022
Meta-RL
Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models and Amortized Policy Search
Qi Wang, Herke van Hoof
NeurIPS 2022
Learning Expressive Meta-Representations with Mixture of Expert Neural Processes
Qi Wang, Herke van Hoof
NeurIPS 2022
RL Theory
Near-optimal regret bounds for multi-batch reinforcement learning
Zihan Zhang, Yuhang Jiang, Yuan Zhou, Xiangyang Ji
COLT 2021
RL Theory
Is reinforcement learning more difficult than bandits? a near-optimal algorithm escaping the curse of horizon
Zihan Zhang, Xiangyang Ji, Simon Du
ICML 2021
RL Theory
Near optimal reward-free reinforcement learning
Zihan Zhang, Simon Du, Xiangyang Ji
ICML 2021
RL Theory
Model-free reinforcement learning: from clipped pseudo-regret to sample complexity
Zihan Zhang, Yuan Zhou, Xiangyang Ji
NeurIPS 2021
RL Theory
Improved variance-aware confidence sets for linear bandits and linear mixture mdp
Zihan Zhang, Jiaqi Yang, Xiangyang Ji, Simon S Du
CVPR 2020
Pfrl: Pose-free reinforcement learning for 6d pose estimation
Jianzhun Shao, Yuhang Jiang, Gu Wang, Zhigang Li, Xiangyang Ji
NeurIPS 2020
RL Theory
Almost optimal model-free reinforcement learningvia reference-advantage decomposition
Zihan Zhang, Yuan Zhou, Xiangyang Ji
NeurIPS 2019
RL Theory
Regret minimization for reinforcement learning by evaluating the optimal bias function
Zihan Zhang, Xiangyang Ji
No publications under current filter conditions.
Blogs
That's all blogs!
No blogs matching current filter.
Team Members
Faculty
 Wang.jpg)
Students
Wutong Xu
Undergraduate Student
Xiang Li
Undergraduate Student
Yuyi Guo
Undergraduate Student
Zhenchan Su
Undergraduate Student
Alumni
Hongchang Zhang
Ph.D. Student (Graduated)
