About

👋🏻 Hi! I’m an incoming Presidential Young Professor at the National University of Singapore (NUS), School of Computing, where I will direct the MAGIC Lab. I am completing my PhD in Computer Science at the University of Washington, co-advised by Prof. Ranjay Krishna and Prof. Dieter Fox. I am also a Graduate Student Researcher at the Allen Institute for AI (AI2), working with the PRIOR and Robotics teams.

My research mission is simple: I teach robots to perceive, reason, and act. I focus on robot learning, embodied AI, and building large-scale robotics foundation models that are deployable in the real world.

Previously, I obtained my B.Eng. in Electrical and Electronic Engineering with Highest Distinction from Nanyang Technological University (NTU), Singapore. I have also interned at NVIDIA as a Research Scientist Intern.

🔥 News

• [2026/05] We are launching MolmoAct2: Action Reasoning Models for Real-World Deployment.
• [2026/04] Our paper MolmoSpaces: Large-Scale Open Ecosystem for Robot Navigation and Manipulation has been accepted to RSS 2026 as Oral.
• [2026/02] Our paper From Mystery to Mastery: Failure Diagnosis and Recovery for Robotic Manipulation has been accepted to ICLR 2026.
• [2026/01] Our paper MolmoAct: Action Reasoning Models that Reason in Space has been accepted to ICRA 2026.
• [2025/12] Honored to join NUS as an inaugural Presidential Young Professor, directing the MAGIC Lab.
• [2025/10] Our paper FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models has been preprinted to arXiv.
• [2025/09] Our paper SAM2Act has been selected as Best Paper Award at RemembeRL @ CoRL 2025.
• [2025/05] Our paper SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation has been accepted to ICML 2025.
• [2025/04] Our paper AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation has been accepted to ICLR 2025.
• [2024/09] Our paper Manipulate-Anything: Automating Real-World Robots using Vision-Language Models has been accepted to CoRL 2024.
• [2024/07] Our papers THE COLOSSEUM and Octopi have been accepted to RSS 2024 as Oral.
• [2024/01] Our paper Selective Visual Representations Improve Convergence and Generalization for Embodied AI has been accepted to ICLR 2024 as Spotlight.
• [2023/10] Our paper NEWTON: Are Large Language Models Capable of Physical Reasoning? has been accepted to EMNLP 2023.
• [2023/07] AR2-D2 accepted to CoRL 2023. Good Time to Ask wins Best Paper Award at Ubiquitous Robots 2023.
• [2022/07] PIP accepted to ECCV 2022. Intuitive Physics Survey accepted to IJCAI 2022 as Oral.

📑 Publications

MolmoAct2: Action Reasoning Models for Real-World Deployment

Haoquan Fang*, Jiafei Duan*, Jason Lee*, Yuquan Deng, Yi Ru Wang, Wilbert Pumacay, Ali Farhadi, Dieter Fox^‡, Ranjay Krishna^‡

arXiv 2026

Paper Code Blog

MolmoAct: Action Reasoning Models that Reason in Space

Jason Lee*^§, Jiafei Duan*^§, Haoquan Fang*^§, Yuquan Deng^§, Shuo Liu^§, Yi Ru Wang^§, Wilbert Pumacay, Ali Farhadi, Dieter Fox^§‡, Ranjay Krishna^§‡

International Conference on Robotics and Automation (ICRA) 2026

Rational Robots @ CoRL 2025 Best Paper Award Runner-up

Data @ CoRL 2025 Best Paper Award Nominee

Paper Code Blog Models Datasets VentureBeat GeekWire

SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation

Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox^‡, Ranjay Krishna^‡, Jiafei Duan^‡

International Conference on Machine Learning (ICML) 2025

RemembeRL @ CoRL 2025 Best Paper Award

Paper Code Project

AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation

Jiafei Duan, Wilbert Pumacay, Nishanth Kumar, Yi Ru Wang, Shulin Tian, Wentao Yuan, Ranjay Krishna, Dieter Fox, Ajay Mandlekar^‡, Yijie Guo^‡

International Conference on Learning Representations (ICLR) 2025

Paper Code Project

Manipulate-Anything: Automating Real-World Robots using Vision-Language Models

Jiafei Duan*, Wentao Yuan*, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox, Ranjay Krishna

Conference on Robot Learning (CoRL) 2024

Paper Code Project

THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation

Wilbert Pumacay*, Ishika Singh*, Jiafei Duan*, Ranjay Krishna, Jesse Thomason, Dieter Fox

Robotics: Science and Systems (RSS) 2024 Oral

Paper Code Project

AR2-D2: Training a Robot Without a Robot

Jiafei Duan, Yi Ru Wang, Mohit Shridhar, Dieter Fox, Ranjay Krishna

Conference on Robot Learning (CoRL) 2023

Paper Code Project

💬 Invited Talks

• [2026] Talk at UWaterloo CS Seminar on "Building Robotics Foundation Models with Reasoning-in-the-Loop" (Host: Victor Zhong).
• [2026] Talk at UT Austin CS Seminar on "Building Robotics Foundation Models with Reasoning-in-the-Loop" (Host: Joydeep Biswas).
• [2026] Talk at University of Michigan CSE Seminar on "Building Robotics Foundation Models with Reasoning-in-the-Loop" (Host: Bernadette Bucher).
• [2026] Talk at MPI for Informatics on "Building Robotics Foundation Models with Reasoning-in-the-Loop" (Host: Christian Theobalt).
• [2025] Keynote at CoRL 2025 Workshop on "Grounded Reasoning from Vision-Language Models".
• [2025] Talk at Mila Robot Learning Seminar on "Towards Robotics Foundation That Can Reason" (Host: REAL Lab).
• [2025] Talk at TTIC Young Researcher Seminar Series on "Towards Robotics Foundation That Can Reason" (Host: Matt Walter).
• [2025] Talk at Georgia Tech on "Towards Robotics Foundation That Can Reason" (Host: GT Institute for Robotics and Intelligent Machines).
• [2025] Talk at Johns Hopkins University on "Towards Robotics Foundation That Can Reason" (Host: Tianmin Shu & Peter Kazanzides).
• [2025] Talk at META FAIR Robotics Group on "Towards Robotics Foundation That Can Reason" (Host: Homanga Bharadhwaj).
• [2024] Talk at Stanford PAIR Group on "Towards Unified Multimodal LLM for Robotics" (Host: Wenlong Huang).
• [2024] Talk at CMU RCHI Group on "Grounded Embodied Intelligence" (Host: Zackory Erickson).
• [2024] Talk at Amazon Lab126 on "Democratizing Robot Learning for All" (Host: Yuyin Sun).
• [2024] Talk at The AI Talks on "Democratizing Robot Learning for All".
• [2024] Talk at UW Allen School Colloquium on "Democratizing Robot Learning for All".
• [2023] Talk at AAAI Summer Symposium on "AR2-D2: Training a Robot Without a Robot".

📖 Academic Services

• Workshop Organizer

  • 3D Vision Language Models (VLMs) for Robotics Manipulation @ CVPR 2025
  • Generalization in Robotics Manipulation Workshop and Challenges @ CVPR 2025
  • Mobile Manipulation: Emerging Opportunities & Contemporary Challenges @ RSS 2025
  • SPACE in Vision, Language, and Embodied AI @ NeurIPS 2025
  • Generalizable Priors for Robot Manipulation @ CoRL 2025

• Conference Reviewer

  • CVPR, ECCV, ICCV, NeurIPS, ICLR, ICML, ICRA, IROS, CogSci
  • RA-L, IEEE Transactions on Automation Science and Engineering, Pattern Recognition

• Teaching

  • UW CSE 571: Robotics — Co-instructor with Prof. Dieter Fox (Winter 2026)
  • UW CSE 571: Robotics — Teaching Assistant (Spring 2025)
  • UW CSE 599H: AI vs Intelligence Augmentation — Head Teaching Assistant (Winter 2023)

🎓 Education

• University of Washington, 2021 - 2026 (expected)

  Doctor of Philosophy in Computer Science

  Co-advised by Prof. Ranjay Krishna and Prof. Dieter Fox

• Nanyang Technological University, 2016 - 2020

  B.Eng. in Electrical and Electronic Engineering

  • Highest Distinction

💼 Experience

• National University of Singapore

  Presidential Young Professor (incoming), 2026 -

  • Directing the MAGIC Lab, School of Computing

• Allen Institute for AI (AI2)

  Graduate Student Researcher, 2023 - 2026

  • Working with the PRIOR team and Robotics team
  • Mentored by Prof. Ranjay Krishna and Prof. Dieter Fox

• NVIDIA

  Research Scientist Intern

  • Research on robotic manipulation and vision-language models

🏅 Awards

• Best Paper Award at RemembeRL @ CoRL 2025 (with SAM2Act)
• Best Paper Award at Ubiquitous Robots 2023 (with Good Time to Ask)
• Spotlight at ICLR 2024 (with Selective Visual Representations)
• Oral at RSS 2024 (with THE COLOSSEUM and Octopi)
• Oral at IJCAI 2022 (with Intuitive Physics Survey)
• Featured in MIT Technology Review, GeekWire, VentureBeat, Business Wire