Tobias Pfaff 🦛
Tobias Pfaff

Research Scientist

Prometheus

About Me

I am a member of technical staff at Prometheus, working on foundation models for physics and engineering. Previously I have worked at NVIDIA, focussing on video world models, and at Google Deepmind, researching a wild mix of topics in RL, robotics, GNNs, physics and video models.

I wasn’t always a ML guy. I used to be a computer graphics researcher, writing fluid, cloth & fracture solvers for SFX; did some geophysics, pondering about permafrost soils in Tibet; worked at a SF startup with a ridiculously bad business idea; and did a stint in video game development as well. To my utter surprise, this weird trajectory has somehow worked out well!

I am also a proud dad, cat owner, and hippo enthusiast. My car is a cargo bike.

Selected Publications

See Google Scholar for the full set of my publications.

Motion prompting: Controlling video generation with motion trajectories.
D Geng, C Herrmann, J Hur, F Cole, S Zhang, T Pfaff, T Lopez, Y Aytar, S Deqing.
CVPR 2025.
PDF Cite Project
Imagen 3.
The Imagen Team.
Whitepaper 2024.
PDF Cite Project
Learning rigid-body simulators over implicit shapes.
Y Rubanova, T Lopez-Guevara, KR Allen, WF Whitney, K Stachenfeld, T Pfaff.
NeurIPS 2024 [oral].
PDF Cite Project
Multiscale meshgraphnets.
M Fortunato, T Pfaff, P Wirnsberger, A Pritzel, P Battaglia.
arXiv 2022.
PDF Cite
Learning mesh-based simulation with graph networks.
T Pfaff, M Fortunato, A Sanchez-Gonzalez, P Battaglia.
ICLR 2021 [best paper].
PDF Cite Code Project
Learning to simulate complex physics with graph networks.
A Sanchez, J Godwin, T Pfaff, R Ying, J Leskovec, P Battaglia.
ICML 2020.
PDF Cite Code Project
More publications
Selected Projects

A selection of projects that I have worked on over the years.

OmniDreams
Imagen 3
Veo
Grim Fandango Remastered
ARCSim
MantaFlow