Modern robots, from assistive and social robots to self-driving cars, are increasingly operating in human-centered environments, requiring them to collaborate more closely with human users than ever before. Beyond performing useful tasks, these robots must also learn and adapt through human interactions to become more effective and intuitive partners. Designing robotic agents that can work alongside, learn from, and adapt to humans is a key research challenge, requiring solutions that address diverse interaction types—including demonstrations, corrections, natural language commands, and preference-based feedback. Another challenge lies in how robots interpret and respond to human input, which can be ambiguous, noisy, and vary across users. Robots must be able to adapt to different interaction styles and evolving user needs while maintaining robust and sample-efficient learning mechanisms. Unlike traditional machine learning settings, where large-scale datasets can be collected and labeled offline, human-in-the-loop learning often requires real-time adaptation with limited supervision, making efficient use of human guidance a critical challenge.
Addressing these challenges demands interdisciplinary collaboration across fields such as machine learning, cognitive science, and control theory. This workshop aims to bring together researchers from robotics, artificial intelligence, machine learning, and human-computer interaction to explore various approaches to human-in-the-loop robot learning. As robots are increasingly deployed in real-world environments, it is more crucial than ever for these agents to be able to correct mistakes on the fly and rapidly personalize their behavior to different users and scenarios. Through discussions on methodologies, benchmarks, and real-world applications, we aim to identify open problems and future directions in human-interactive robot learning.
| Start Time (PDT) | End Time (PDT) | Event |
|---|---|---|
| 8:20 AM | 8:30 AM | Opening Remarks |
| 8:30 AM | 09.50 AM | Invited Speakers: Part 1 Luka Peternel - Incorporating Human Physical Behaviour into Robot Control Loop for Adaptive Human-Robot Co-Manipulation (8.30-8.50)
The talk will focus on how to incorporate human physical behaviour into the robot control loop for adaptive human-robot co-manipulation and co-learning.
The human physical behaviour is encoded with several approaches. Higher-level human physical behaviour is learned with a statistical machine learning
method to capture the stochasticity in human actions. The learned behaviour is also augmented by human motor control models that provide additional
intent prediction capabilities to the collaborating robot by understanding the underlying tradeoffs in human movements. The insights about the
lower-level human physical behaviour are incorporated via musculoskeletal models to ensure ergonomics and safety of co-manipulation. Finally,
a co-learning method based on reinforcement learning facilitates continuous adaptation of both the robot and the human. For a more direct
transfer of human skill, we will examine the concept of teaching robots through teleimpedance, where the developed interfaces enable a
human operator to (remotely) teach and correct physically interactive tasks.
Harold Soh - Thoughts on Teaching Robots: Diffusion Models, Test-Time Conditioning, and the Path Ahead (8.50-9.10)
In this talk, I will present our recent work on imitation learning using diffusion models, with
a focus on test-time conditioning to adapt robot behavior. I will also highlight our use of differentiable
model checking to validate behavioral correctness. Time permitting, I hope to open a discussion on the
limitations of current approaches—including our own—and the shifts required to advance foundation model-based
methods for human-centered robot teaching.
Tapomayukh Bhattacharjee - Towards Robot Caregiving: Building Robots That Work with Humans-in-the-Loop (9.10-9.30)
How can we build robots that meaningfully assist people with mobility limitations in their daily lives?
To support complex caregiving tasks such as robot-assisted feeding, transferring, bathing, and meal preparation,
robots must physically interact with people and objects in dynamic, unstructured environments where
robot autonomy can fail. In this talk, I will present our work on building human-in-the-loop physical
robot caregiving systems that integrate multimodal perception with adaptive algorithms, enabling them
to be used by non-expert users and to adapt their strategies to unique user preferences. Together,
these efforts move us closer to creating caregiving robots that are not only technically capable, but
are also responsive to the real needs of people in care settings.
Matthew Gombolay - Teaching the Teachers: Robots Helping Humans Demonstrate for Better Robot Learning (9.30-9.50)
Assistive robots hold a promise of addressing critical societal needs with an aging
population, such as supporting older adults with activities of daily living. Yet, current systems are
costly, narrowly focused, and unscalable. While large-scale datasets and generalist policies
show promise, achieving positive societal impact will require robots to learn directly from
diverse, non-expert users, who often lack expertise as robot teachers to provide effective data.
In this talk, I will share findings from studies with individuals with Mild Cognitive Impairment
(MCI) and their caregivers, offering unique insights into the challenges and design principles for
user-driven robot learning. I will also present approaches to help end-users provide better
demonstrations, including a reciprocal teaching framework where robots model user behaviors
and offer real-time feedback. Finally, I will propose a roadmap for removing roboticists from the
loop, enabling scalable, autonomous deployment of cognitive robots in homes.
|
| 9:50 AM | 10.30 AM | Paper Presentations: Part 1 |
| 10:30 AM | 11:30 AM | Coffee Break & Poster Session |
| 11:30 AM | 12:30 PM | Panel Discussion - Will robotics achieve zero-shot generalization or will we always need to fine-tune with human feedback? Host: Mike Hagenow Panelists: Harold Soh, Tapomayukh Bhattacharjee, Daniel Brown, Karl Pertsch |
| 12:30 PM | 2:30 PM | Lunch |
| 2:30 PM | 4:00 PM | Invited Speakers: Part 2 Andreea Bobu - Reading Between the Lines: Using Language Models to Amplify Human Input in Robot Learning (2.30-2.50)
Human-in-the-loop robot learning faces a fundamental data challenge that general machine learning
doesn't: unlike settings where we can collect massive offline datasets, robots must learn from limited,
real-time human interactions. This creates a critical bottleneck: we need methods that can make the
most of limited human input, or, in other words, that can learn a lot from a little. The key insight
in this talk is that large language models, having been trained on vast amounts of human data, already
possess the common sense and semantic priors we need to fill in these gaps. When someone demonstrates
a task or gives feedback, there's often implicit information that seems obvious to humans but that
robots overlook completely. I discuss three approaches that use language models to "read between the
lines" of human input. I demonstrate how LLMs can take sparse human labels and enable robots to
generalize to complex expressions, extract hidden preferences that are implied by human behavior
but not explicitly stated, and identify missing task concepts based on the situational context of
human input. By strategically combining minimal human input with the rich prior knowledge embedded
in language models, we can achieve the kind of sample-efficient learning that human-in-the-loop
robotics demands for real-world deployment.
Elliott Rouse - Preliminary results on sim-to-real RL control for the Open-Source Leg v2 (2.50-3.10)
TBD.
Katia Sycara - Human-AI collaboration and goal alignment in planning for dynamic environments (3.10-3.30)
We are interested in collaborative planning between agents that learns and humans in
environments that are dynamic and time-stressed. In such a setting, plan generation may
involve multiple iterations of human feedback to an AI in order to effectively capture the
human's preferences for various plan constraints, or when plans need to be revised . This
topic presents two major challenges relevant to this talk. First, Reinforcement Learning,
the natural framework for sequential decision making, produces black box policies,
making it very challenging (impossible) for humans to interact with it in bidirectional ways.
Second, Reinforcement Learning has shown to be vulnerable to mis-specification of the
reward function resulting in mis-alignment of the reward with human intended goal(s).
Finally, Reinforcement Learning policies determine actions to perform in a given
environment state during execution; extracting long-term plans from policies is difficult,
especially where environments are highly uncertain, or where a model of environment
dynamics is unavailable. To address these challenges we have developed a neuro-
symbolic approach where Large Language Models (LLMs) are used to translate user
feedback to machine-understandable goal specifications for interpretable interaction
between human and agent. Creating suitable machine understandable descriptions of the
planning domain, problem, and goal requires expertise in the planning language, limiting
the utility of these tools for non-expert humans. Our approach performs initial translation
of goal specifications to a set of Planning Domain Definition Language (PDDL) goal
constraints using an LLM; such translations often result in imprecise symbolic
specifications, which are difficult to validate directly. To address this, we use an
evolutionary approach to generate a population of symbolic goal specifications and train a
LSTM model to assess whether induced plans in the population adheres to the natural
language user goal specifications and feedback. I will present our approach, evaluation
results and discuss open challenges.
Kay Ke - From Teachers to Supervisors: Human Roles in Robot Learning (3.30-3.50)
Humans have always played a central role in the development of robot learning systems—providing
data, guiding training, and shaping evaluation. In this talk, I revisit the often-underappreciated
contributions of human involvement by categorizing these roles into three key types: teachers,
experts, and supervisors. I will explore how each role influences the construction of robot
foundation models and examine how different forms of human input affect model performance.
Through this lens, I aim to highlight the critical, evolving relationship between human guidance
and scalable robot learning—and invite the community to collectively advance the design of scalable
human-in-the-loop systems.
|
| 4:00 PM | 5:00 PM | Coffee Break & Poster Session |
| 5:00 PM | 5.40 PM | Paper Presentations: Part 2 |
| 5:40 PM | 6:30 PM | Panel Discussion - Can we replace humans’ feedback with feedback from foundation models? Host: Erdem Bıyık Panelists: Katia Sycara, Andreea Bobu, Maegan Tucker, Kay Ke |
