Title:	Joint Diffusion Model for Reinforcement Learning
DNr:	Berzelius-2023-236
Project Type:	LiU Berzelius
Principal Investigator:	Ruoqi Zhang <ruoqi.zhang@it.uu.se>
Affiliation:	Uppsala universitet
Duration:	2023-09-12 – 2024-04-01
Classification:	20202
Keywords:

Abstract

In joint machine learning, multiple tasks or objectives are learned simultaneously. The motivation behind this is that the tasks are often interrelated, and learning them together can enhance the performance on individual tasks. In [1] the author proposes to combine standard diffusion models with classifiers by sharing the parameterization. Their experimental results show that diffusion model representations are useful for improving the classification prediction result. This is the power of the shared representation and its regularization effect. Our idea is to use this power on image-based reinforcement learning [2], where $p(q\vert s,a)$ is the standard on the top of generator $p(s)$ and actor generator $\pi(a\vert s)$. Hopefully, the shared representation can improve the generalization to overcome the "out-of-distribution actions and states" problems and learn more accurate q-values. 1] Kamil Deja, Tomasz Trzcinski, and Jakub M Tomczak. Learning data representations with joint diffusion models. arXiv preprint arXiv:2301.13622 [2] Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, and Chelsea Finn. Offline reinforcement learning from images with latent space models. In Learning for Dynamics and Control, pages 1154–1168. PMLR, 2021.