Yiming Huang

I'm a second-year master’s student in Robotics at GRASP Lab, University of Pennsylvania. I received my bachelor's degree in Mathematics and Computer Science from NYU Shanghai.

My interests revolve broadly around the fields of vision, language, and human-robot interaction with a focus on the perception of human behavior and 3D environments. I am currently working on egocentric vision for the analysis of hand-eye coordination patterns in everyday tasks under the supervision of Prof. Jianbo Shi. I am also working on improving the controllability of text to 3D human motion generation under the supervision of Prof. Lingjie Liu and Prof. Mark Yatskar.

profile photo

Email  /  GitHub  /  Google Scholar  /  LinkedIn

profile photo profile photo


(* indicates equal contribution)

project image

DiffusionPhase: Motion Diffusion in Frequency Domain

Weilin Wan* , Yiming Huang*, Shutong Wu, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu
In Submission
arxiv / video / code / website /

We propose a network encoder that converts motion sequences into periodic signals and a conditional diffusion model for predicting periodic motion parameters based on text descriptions and the starting pose, enabling the generation of a broader variety of high-quality longer motion sequences.

project image

Ego-Exo4D: Understanding Skilled Human Activity from First-and Third-Person Perspectives

Kristen Grauman, Andrew Westbury, (et al., including Yiming Huang)
In Submission
arxiv / video / website /

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge, centered around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair).

project image

Gaze-Hand-Object Interaction in Egocentric Videos

Principal Investigator: Prof. Jianbo Shi

Automatic retrieval and 3D reconstruction of interactions between hand, object, and gaze behavior in egocentric and multi-view video data.

project image

Controllability and Interpretability for Text-to-Motion Generation using LLMs

Principal Investigators: Prof. Lingjie Liu and Prof. Mark Yatskar

Applying LLMs to improve the controllability and interpretability of text-to-motion generation.

project image

Investigating Fiducial Marker Network Characteristics for Mobile Indoor Robot Navigation

Yiming Huang, Bharadwaj R. K. Mantha, Borja Garcia de Soto
NYU Shanghai Undergraduate Research Symposium , 2020
poster /

Implemented and evaluated a protocol for optimizing April Tag placement during automatic marker-based navigation of a Turtlebot3 Waffle Pi platform in Gazebo simulation.


project image

SICK TiM$10K University Challenge 2022

University of Pennsylvania
12/2022 - 04/2024

Sparse 3D Reconstruction Module with TIM8xx 2D LiDAR, programmed in Python and Lua.

project image

Recovery RL for Quadroter Navigation

UPenn ESE 650: Learning in Robotics
02/2023 - 04/2023
code /

Extension of Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones” by Thananjeyan et. al to 3D Quadroter navigation tasks.

project image

Quadroter Maze Navigation

UPenn MEAM 620: Advanced Robotics
01/2023 - 04/2023

Implemented a PD Controller, A* Trajectory Planner, min-jerk trajectory generator and VIO state estimator for navigating a CrazyFlie 2.0 quadroter in maze environments.

project image

Robot Arm Block Stacking Challenge

UPenn MEAM 520: Intro to Robotics
09/2022 - 12/2022

Developed manipulation and planning strategies to control a 7 DOF Franka-Emika Panda manipulator arm with vision sensors for April Tag detection to compete in a block stacking challenge.

project image

PIC Math Program 2020

New York University and Metropolitan Transit Authority New York City Transit
01/2020 - 05/2020
code / slides /

Graph-based Automatic Task Assignment System for Bus Fare Evasion Data Collection.

project image

Bittle Path Finding

Petoi Bittle Project
07/2021 - 08/2021
code /

Path finding and navigation algorithms in a grid environment for the Petoi Robot Dog Bittle


Graduate Teaching Assistant at the University of Pennsylvania:

  • Fall 2023, Spring 2023, Fall 2022: CIT5900 Programming Languages and Techniques
  • Fall 2023, Spring 2023: CIS4190/5190 Applied Machine Learning
  • Summer 2023: ESE 5410 Machine Learning for Data Science
  • Course Development at the University of Pennsylvania:

  • EAS 5740 How to Use Data
  • Undergraduate Teaching Assistant at NYU Shanghai

  • Summer 2022, Spring 2019: CSCI-SHU 101 Intro to Computer Science
  • Spring 2022: CSCI-SHU 220 Algorithms
  • Spring 2022, Fall 2021: CSCI-SHU 2314 Discrete Mathematics
  • Fall 2021, Fall 2018: CSCI-UA 9002/CSCI-SHU 11 Intro to Computer Programming
  • Spring 2021: CENG-SHU 202 Computer Architecture
  • Fall 2020: MATH-SHU 235 Probability and Statistics
  • Other:

    2021-2022: Fablab O Shanghai STEAM Instructor


    I enjoy doodling! doodle

    I'm also a member of the Penn Enchord Acapella Group, check out some of our performances!

    Here's a small stash of some of my acapella arrangments :)

    Sourced from this and this template.