Yiming Huang

I'm a second-year master’s student in Robotics at GRASP Lab, University of Pennsylvania. I received my bachelor's degree in Mathematics and Computer Science from NYU Shanghai.

My interests revolve broadly around the fields of vision, language, and human-robot interaction with a focus on the perception of human behavior and 3D environments. I am currently working on egocentric vision for the analysis of hand-eye coordination patterns in everyday tasks under the supervision of Prof. Jianbo Shi. I am also working on improving the controllability of text to 3D human motion generation under the supervision of Prof. Lingjie Liu and Prof. Mark Yatskar.

Email / GitHub / Google Scholar / LinkedIn

Research

(* indicates equal contribution)

	DiffusionPhase: Motion Diffusion in Frequency Domain Weilin Wan* , Yiming Huang, Shutong Wu, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu In Submission* arxiv / video / code / website / We propose a network encoder that converts motion sequences into periodic signals and a conditional diffusion model for predicting periodic motion parameters based on text descriptions and the starting pose, enabling the generation of a broader variety of high-quality longer motion sequences.
	Ego-Exo4D: Understanding Skilled Human Activity from First-and Third-Person Perspectives Kristen Grauman, Andrew Westbury, (et al., including Yiming Huang) In Submission arxiv / video / website / We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge, centered around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair).
	Gaze-Hand-Object Interaction in Egocentric Videos Principal Investigator: Prof. Jianbo Shi Ongoing Automatic retrieval and 3D reconstruction of interactions between hand, object, and gaze behavior in egocentric and multi-view video data.
	Controllability and Interpretability for Text-to-Motion Generation using LLMs Principal Investigators: Prof. Lingjie Liu and Prof. Mark Yatskar Ongoing Applying LLMs to improve the controllability and interpretability of text-to-motion generation.
	Investigating Fiducial Marker Network Characteristics for Mobile Indoor Robot Navigation Yiming Huang, Bharadwaj R. K. Mantha, Borja Garcia de Soto NYU Shanghai Undergraduate Research Symposium , 2020 poster / Implemented and evaluated a protocol for optimizing April Tag placement during automatic marker-based navigation of a Turtlebot3 Waffle Pi platform in Gazebo simulation.

Projects

	SICK TiM$10K University Challenge 2022 University of Pennsylvania 12/2022 - 04/2024 Sparse 3D Reconstruction Module with TIM8xx 2D LiDAR, programmed in Python and Lua.
	Recovery RL for Quadroter Navigation UPenn ESE 650: Learning in Robotics 02/2023 - 04/2023 code / Extension of Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones” by Thananjeyan et. al to 3D Quadroter navigation tasks.
	Quadroter Maze Navigation UPenn MEAM 620: Advanced Robotics 01/2023 - 04/2023 Implemented a PD Controller, A* Trajectory Planner, min-jerk trajectory generator and VIO state estimator for navigating a CrazyFlie 2.0 quadroter in maze environments.
	Robot Arm Block Stacking Challenge UPenn MEAM 520: Intro to Robotics 09/2022 - 12/2022 Developed manipulation and planning strategies to control a 7 DOF Franka-Emika Panda manipulator arm with vision sensors for April Tag detection to compete in a block stacking challenge.
	PIC Math Program 2020 New York University and Metropolitan Transit Authority New York City Transit 01/2020 - 05/2020 code / slides / Graph-based Automatic Task Assignment System for Bus Fare Evasion Data Collection.
	Bittle Path Finding Petoi Bittle Project 07/2021 - 08/2021 code / Path finding and navigation algorithms in a grid environment for the Petoi Robot Dog Bittle