Robot Learning in the Era of Foundation Models

Schedule

Class meets Mondays. Topics are tentative and subject to change.

Week	Date	Topic
1	Mon, Aug 10, 2026	Course overview & introduction to embodied AI
2	Mon, Aug 17, 2026	Robotic manipulation foundations I — rigid-body kinematics & contact mechanics
3	Mon, Aug 24, 2026	Robotic manipulation foundations II — grasping & motion planning
4	Mon, Aug 31, 2026	Imitation learning I — behavior cloning & DAgger
5	Mon, Sep 7, 2026	Imitation learning II — Action Chunking Transformers (ACT) & Diffusion Policy
6	Mon, Sep 14, 2026	Reinforcement learning I — MDPs & value-based methods
—	Sep 19 – Sep 27, 2026	Recess / reading & examination period — no class
7	Mon, Sep 28, 2026	Reinforcement learning II — policy-gradient methods, PPO & SAC
8	Mon, Oct 5, 2026	Reinforcement learning III — offline RL & sim-to-real transfer
9	Mon, Oct 12, 2026	Vision-Language-Action models I — RT-1, RT-2, OpenVLA
10	Mon, Oct 19, 2026	Vision-Language-Action models II — MolmoAct, π-0.5, cross-embodiment learning
11	Mon, Oct 26, 2026	Long-horizon reasoning — SayCan, Manipulate-Anything, Code-as-Policies
12	Mon, Nov 2, 2026	Foundation models for robotics — world models & generative simulators
13	Mon, Nov 9, 2026	Course wrap-up & project work session
14	Nov 14 – Nov 20, 2026	Reading week — no class
—	Mon, Nov 23, 2026	Final project presentations

Project

Students complete a quarter-long capstone project in teams of two to three, applying course concepts to an open problem in robot learning — e.g. imitation or reinforcement learning for manipulation, adapting a Vision-Language-Action model to a new task or embodiment, or evaluating a foundation model's capability as a robot policy or world model. Projects may be validated in simulation or on physical hardware where available.

Milestone	Deliverable
Proposal	1-page problem statement, related work, and planned approach
Check-in	Progress update & preliminary results
Final	Presentation and written report

Commentaries

Reading Commentaries

Most weeks include reading requirements featuring news articles and academic conference papers selected to illustrate key course concepts for class discussion. Students must write commentaries describing their takeaways from papers, due 48 hours before the discussion section.

All submissions go to Canvas.

Commentary Guidelines

The purpose is to encourage critical thinking about research importance, not summarization. Appropriate topics include:

• Future research directions inspired by the papers
• Assessment of paper importance
• Novel or questionable methodology observations
• Effectiveness of the papers' messaging
• How papers changed your perspective

A recommended structure uses two positive points and one criticism per paper (approximately 4 sentences each).

Deadline: Canvas submission at 1:30pm, two days before the discussion session. Late submissions are not accepted. After submission, commentaries become available to discussion leaders.

Grading Scale: Check-minus/Check/Check-plus

• Check-minus: “Surface-level engagement with the readings, or a repeat of a style of critique” that instructors discouraged, including personal preference statements or paper summaries rather than reflective analysis
• Check: Effective engagement demonstrating comprehension with nontrivial observations
• Check-plus: Rare instances of exceptional insight worth sharing in discussion

Discussion Guidelines

Format adapted from Colin Raffel's role-playing seminar and the CSE 599H commentaries page.

Seminar Roles & Responsibilities

1. 🎤 Discussion Leader (presents first)
   You have access to all commentaries submitted by other students 48 hours before class. Before the session, read the paper and all submitted responses. In class, open with an 8-minute summary covering: (a) the paper's core problem, method, and claimed contributions, (b) a synthesis of the main themes, agreements, and disagreements across the submitted commentaries. After your summary, spur discussion by posing 2–3 open questions drawn from tensions or gaps you noticed in the responses. Throughout the session, you help moderate — inviting quieter roles in and connecting related points across presenters.
2. 👩🏽‍🔬 Scientific Peer Reviewer
   The paper has not been published yet and is currently under submission to a top robotics venue where you've been assigned as a reviewer. Complete a full review answering all prompts of the official review form for RSS, CoRL, ICRA, or IROS (whichever best fits the paper), including robotics-specific criteria such as real-robot validation and reproducibility of the hardware setup. Conclude with a clear accept/reject recommendation and the changes you would require for acceptance.
3. 🏺 Archaeologist
   This paper was found buried in the desert. You must determine where it sits in the lineage of the field. Find and report on one older paper cited within it that substantially influenced the work (e.g., tracing a modern learning-based manipulation paper back to classical control or motion planning), and one newer paper that cites it. Explain what each connection reveals about how the ideas evolved.
4. 🍜 Academic Researcher
   You're a researcher starting a new project in this area. Propose an imaginary follow-up project that is not just related to the current paper, but only possible because of the existence and success of the current paper. Be prepared to sketch the research question, approach, and what you'd need to make it work.
5. 🔍 Hardware Skeptic (Experimental Auditor)
   You don't trust the results until you've audited them. Scrutinize the evaluation: How many trials were run? What's the split between real-robot runs and simulation? Were failure cases reported, and how were failures counted? Was the baseline given a fair tuning budget? Does the demo video appear cherry-picked (single takes, controlled environments, hidden resets)? Report which claims you find well-supported, which are shaky, and what one additional experiment would most strengthen the paper.

Assignments

One hands-on in-class assignment accompanies the course, implementing and evaluating a policy-learning baseline (e.g. behavior cloning or a diffusion policy) on a manipulation benchmark. Submissions are graded on correctness, code quality, and a brief write-up of results. Due date TBD.

Grading

Component	Weight
Class Participation	30%
Project Proposal (Group)	20%
Final Project Proposal (Group)	25%
Project Report (Individual)	15%
In-Class Assignment	10%

Demo

Live 3D simulation of a robotic arm picking and placing cubes — drag to orbit, scroll to zoom.

Blue-tinted zone: pick tray · orange-tinted zone: place tray. The arm solves inverse kinematics live for every move.

Robot Learning Reading List

Core papers and texts for each unit. Updated as the syllabus firms up.

Foundations & preliminaries

• Reinforcement Learning: An Introduction
• Modern Robotics
• Probabilistic Robotics
• Foundations of Computer Vision
• Artificial Intelligence: A Modern Approach
• Robot Learning in the Era of Foundation Models: A Survey

Imitation learning & behavior cloning

• ALVINN: An Autonomous Land Vehicle in a Neural Network
• End-to-End Training of Deep Visuomotor Policies
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
• Generative Adversarial Imitation Learning
• Implicit Behavioral Cloning
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
• Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation
• RT-1: Robotics Transformer for Real-World Control at Scale
• Transporter Networks: Rearranging the Visual World for Robotic Manipulation
• CLIPort: What and Where Pathways for Robotic Manipulation

Reinforcement learning

• Trust Region Policy Optimization (TRPO)
• Proximal Policy Optimization (PPO)
• Soft Actor-Critic (SAC)
• Deep Deterministic Policy Gradient (DDPG)
• Solving Rubik's Cube with a Robot Hand
• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

Sim-to-real transfer

• Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
• Eureka: Human-Level Reward Design via Coding Large Language Models
• Learning agile and dynamic motor skills for legged robots
• 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation
• MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Motion Policy Networks
• Factory: Fast Contact for Robotic Assembly
• GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Representation learning & perception for robotics

• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• Where2Act: From Pixels to Actions for Articulated 3D Objects
• RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
• R3M: A Universal Visual Representation for Robot Manipulation
• Masked Visual Pretraining for Motor Control (MVP)
• VIP: Universal Visual Reward and Representation
• DINOv2: Learning Robust Visual Features without Supervision
• Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

Robotics foundation models

• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• OpenVLA: An Open-Source Vision-Language-Action Model
• π0.5: a Vision-Language-Action Model with Open-World Generalization
• A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
• Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets
• MolmoAct2: Action Reasoning Models for Real-World Deployment
• Gemini Robotics: Bringing AI into the Physical World
• World Action Models are Zero-shot Policies
• RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

Simulation for robot learning

• Coppelia Simulation
• ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
• Isaac Lab
• Drake
• MuJoCo
• MolmoSpace
• Genesis World
• RoboSuite

Benchmarks & evaluation

• RLBench: The Robot Learning Benchmark & Learning Environment
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
• THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
• RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
• BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
• RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies

Datasets for robot learning

• Open X-Embodiment: Robotic Learning Datasets and RT-X Models
• BridgeData V2: A Dataset for Robot Learning at Scale
• ABC: Scalable Behavior Cloning with Open Data, Training and Evaluation
• MolmoAct2: Action Reasoning Models for Real-World Deployment
• MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation
• BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
• DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
• AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems