Robots of the future should have the ability to quickly learn new tasks in an interactive manner.  Reinforcement learning (RL) is a common approach for teaching robots to do difficult control tasks, but it requires tedious hand-designing of reward functions. At PoRTaL, I'm investigating how reward functions can be inferred from video demonstrations. Specifically, I have found that existing sequence-based distance functions result in reward hacking due because they do not always respect the order of subgoals in the video demonstration. I am researching a more principled, temporally consistent approach to sequence-matching for RL. In the future, this can be used to help robots learn a diverse set of tasks from the wide variety of human videos available on the internet. 

Publications and Presentations

Huaxiaoyue Wang*, William Huey*, Anne Wu, Yoav Artzi, Sanjiban Choudhury.  "Time Your Rewards: Learning Temporally Consistent Rewards from a Single Video Demonstration." CoRL 2024 Workshop on Whole-body Control and Bimanual Manipulation.

* Indicates equal contribution. 

Modern mobile robots need to navigate complex environments with boundless types of obstacles and terrain. Currently, most traversability prediction methods rely on heuristics, human demonstration, or pretraining on a specific set of object classes. At the Robotic Systems Lab, I investigated applications of large pretrained vision language models to traversability prediction.

This project involved implementing existing traversability analysis methods in simulation, coming up with a novel approach to traversability analysis using visual semantics, developing a ROS package to run the method interactively on an Anymal D robot dog, and designing experiments to ablation test the method. We demonstrated long horizon unguided exploration using  zero shot traversability prediction. Additionally, we investigated how traversability knowledge can act as a prior for imitation learning. 

Publications and Presentations

William Huey*, Sean Brynjolfsson*, Donald Greenberg. "Distilling Vision-Language Models for Real-Time Traversability Prediction". Cornell Discover Undergraduate Research in Engineering Showcase, 2024. Poster can be found here 

* Indicates equal contribution. 

A major problem in mobile robotics is predicting what parts of a given terrain will be traversable for a robot, and planning to avoid untraversable areas. My most recent project with the Greenberg Lab is SuperEdge, a self-supervised learning method for traversability analysis from partial environment observations. As part of my final project for CS 4756: Robot Learning at Cornell, I made a video with my findings.

To support this project, I developed a python library containing end-to-end motion planning algorithms that interface with legged robots in Nvidia Isaac Sim. It includes traversability estimation, environment graph generation, graph search, and path tracking. I also implemented and benchmarked a variety of supervised and reinforcement learning methods for a quadruped to perform obstacle avoidance tasks in Isaac Sim. These included behavioral cloning, DAgger, q-learning, and actor-critic policy gradient. 

As a research intern at NASA IV&V, I was tasked with applying combinatorial testing methods to HLS Starship flight control software. In collaboration with verification researchers at NIST, systems engineers at NASA, and software engineers at SpaceX, I surveyed the current methods for system-level verification, and proposed a workflow that would integrate combinatorial testing.

In order to improve the efficiency of this workflow and make it viable for use by engineers, I developed a desktop application that would automatically generate unit tests with a specified degree of combinatorial coverage. This required me to design a graph optimization algorithm to find the most efficient ordering of tests in state based systems. Ultimately, the application was able to suggest tests that would improve coverage, and it significantly increased the efficiency of combinatorial coverage analysis.

Atomic Force Microscopy (AFM) allows for measurements of certain properties of materials at nanometer scale. Nanomachining by Regressively Actuated Setpoint (NRASP) is a novel method that uses an AFM to precisely machine any pattern into the surface of the material. I designed and implemented the NRASP algorithm, which uses linear regression and gaussian smoothing to calculate and apply an optimal voltage to the material at different locations. In the process of interfacing with the AFM, I found a bug in its source code that had existed for over 10 years. This method is currently being used to reveal hidden properties of ferroelectric materials that vary with depth and location.

Publications and Presentations

Coming soon...