Ruixuan Liu

PhD Student · Robotics Institute · Carnegie Mellon University

I am a PhD student in the Robotics Institute in the School of Computer Science at Carnegie Mellon University, advised by Prof. Changliu Liu (Intelligent Control Lab). My research focuses on robot learning, manipulation, and control; generative intelligence; and human-robot collaboration.

I received my Bachelor's degree in Electrical and Computer Engineering, with a minor in Robotics, from Carnegie Mellon University. As an undergraduate, I was fortunate to work in the AirLab led by Prof. Sebastian Scherer, where I worked on sensor fusion and 3D reconstruction for structural inspection.

Photo of Ruixuan Liu

Highlights

BrickGPT won the Best Paper Award (Marr Prize) at ICCV 2025!
BrickGPT won the Best Paper Award (Marr Prize) at ICCV 2025!
Won First Prize in the 2025 ICRA What Bimanuals Can Do (WBCD) competition!
Won First Prize in the 2025 ICRA What Bimanuals Can Do (WBCD) competition!
Demonstrated safe interactive industrial robots to the U.S. President.
Demonstrated safe interactive industrial robots to the U.S. President.

News

Show older news
  • 2025/09Our work on Physical AI was featured in CMU News.
  • 2025/07BrickGPT was featured in CMU News.
  • 2025/05Our team won First Prize ($60k) in the 2025 ICRA What Bimanuals Can Do (WBCD) competition! [News]
  • 2024/11Our work on generative manufacturing and robotic assembly was featured in Modern Machine Shop.
  • 2024/09Finished my internship as an Applied Scientist at Amazon Robotics.
  • 2023/10Our work on robotic brick construction was featured in CMU News.
  • 2022/12Demonstrated our work on proactive human-robot co-assembly to Ford. [CMU News]
  • 2022/08Finished my internship as a Research Scientist at Siemens Berkeley.
  • 2022/01Demonstrated our work on safe HRC with interactive industrial robots to the U.S. President at Mill 19, Pittsburgh. [President Biden's Twitter] [CMU News] [Lab Website]

Research

I'm interested in robotics, particularly manipulation, generative intelligence, and human-robot collaboration. Selected publications are listed below.

Robustifying Long-term Human-Robot Collaboration through an Intent- and Progress-Aware Framework

Robustifying Long-term Human-Robot Collaboration through an Intent- and Progress-Aware Framework

Peiqi Yu, Yuner Zhang, Abulikemu Abuduweili, Jianlin Dou, Ruixuan Liu, Changliu Liu

IFAC Workshop on Cyber-Physical and Human Systems (CPHS), 2026

We introduce a hierarchical, multimodal framework for efficient long-term human-robot collaboration that combines visual observations and speech commands for intuitive interaction. Deployed on a Kinova Gen3 robot, the framework is validated through extensive real-world user studies.

Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking

Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking

Ruixuan Liu, Alan Chen, Weiye Zhao, Changliu Liu

IEEE Robotics and Automation Letters (RA-L), 2025

We formulate combinatorial assembly sequence planning as a reinforcement learning problem. An optimization-based, physics-aware action mask tackles the sim-to-real gap and combinatorial complexity, guiding policy learning and ensuring violation-free deployment by planning physically executable assembly sequences.

Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts

Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts

Hongyi Chen, Yunchao Yao, Ruixuan Liu, Changliu Liu, Jeffrey Ichnowski

American Control Conference (ACC), 2025

We show that optimizing visual and text prompts improves the spatial reasoning of VLMs, enabling them to act as black-box controllers for motion-level correction and task-level recovery from unknown failures. Key visual elements are highlighted in both prompts, and reasoning is decomposed into failure detection and control generation.

Robotic Planning under Hierarchical Temporal Logic Specifications

Robotic Planning under Hierarchical Temporal Logic Specifications

Xusheng Luo, Shaojun Xu, Ruixuan Liu, Changliu Liu

IEEE Robotics and Automation Letters (RA-L), 2024

We propose a decomposition-based method for multi-robot planning under hierarchical temporal logic specifications. Each specification is broken into temporally related sub-tasks, relations among sub-tasks across the hierarchy are mined, and a mixed-integer linear program generates a spatio-temporal plan for each robot.

A Lightweight and Transferable Design for Robust LEGO Manipulation

A Lightweight and Transferable Design for Robust LEGO Manipulation

Ruixuan Liu, Yifan Sun, Changliu Liu

International Symposium on Flexible Automation (ISFA), 2024

We make robotic LEGO manipulation safe and efficient through hardware-software co-design. A custom end-of-arm tool reduces the problem dimension so that large industrial robots can easily manipulate LEGO bricks, and evolution strategies safely optimize the robot’s motion.

GUARD: A Safe Reinforcement Learning Benchmark

GUARD: A Safe Reinforcement Learning Benchmark

Weiye Zhao, Rui Chen, Yifan Sun, Ruixuan Liu, Tianhao Wei, Changliu Liu

Transactions on Machine Learning Research (TMLR), 2024

We introduce GUARD, a Generalized Unified SAfe Reinforcement Learning Development Benchmark. GUARD covers a wide variety of RL agents, tasks, and safety constraints, includes self-contained implementations of state-of-the-art safe RL algorithms, and is highly customizable in both tasks and algorithms.

Simulation-aided Learning from Demonstration for Robotic LEGO Construction

Simulation-aided Learning from Demonstration for Robotic LEGO Construction

Ruixuan Liu, Alan Chen, Xusheng Luo, Changliu Liu

arXiv, 2023

We present SaLfD, a simulation-aided learning-from-demonstration framework that makes it easy to deploy LEGO prototyping on robots. The robot watches a user build a custom object, extracts a construction plan, and verifies both the plan and the resulting prototype in simulation.

Robotic LEGO Assembly and Disassembly from Human Demonstration

Robotic LEGO Assembly and Disassembly from Human Demonstration

Ruixuan Liu, Yifan Sun, Changliu Liu

ACC Workshop on Recent Advancement of Human Autonomy Interaction and Integration, 2023

We study automatic prototyping with LEGO bricks. To meet individual needs and support sustainability, we present a framework that learns both assembly and disassembly sequences from human demonstrations, so robots can build customized designs and take them apart for reuse.

Proactive Human-Robot Co-Assembly: Leveraging Human Intention Prediction and Robust Safe Control

Proactive Human-Robot Co-Assembly: Leveraging Human Intention Prediction and Robust Safe Control

Ruixuan Liu, Rui Chen, Abulikemu Abuduweili, Changliu Liu

IEEE Conference on Control Technology and Applications (CCTA), 2023

We present an integrated framework for proactive human-robot collaboration. A robust intention prediction module, trained with prior task knowledge and human-in-the-loop data, guides the robot, while robust safe control ensures safety under uncertainty. We demonstrate the framework on a co-assembly task with a Kinova Gen3 robot.

Zero-shot Transferable and Persistently Feasible Safe Control for High Dimensional Systems by Consistent Abstraction

Zero-shot Transferable and Persistently Feasible Safe Control for High Dimensional Systems by Consistent Abstraction

Tianhao Wei, Shucheng Kang, Ruixuan Liu, Changliu Liu

IEEE Conference on Decision and Control (CDC), 2023

We propose a system abstraction method that lets energy functions for safe control be designed on a low-dimensional model while guaranteeing persistent feasibility. The resulting safe controller transfers zero-shot to any system with the same abstraction, for example, a robot arm holding different tools.

Safe Interactive Industrial Robots using Jerk-based Safe Set Algorithm

Safe Interactive Industrial Robots using Jerk-based Safe Set Algorithm

Ruixuan Liu, Rui Chen, Changliu Liu

International Symposium on Flexible Automation (ISFA), 2022

We introduce JSSA, a jerk-based safe set algorithm that ensures collision avoidance while respecting robot dynamic constraints. It extends the original safe set algorithm, previously limited to second-order systems with unbounded acceleration, and is validated on human-robot interaction tasks with a FANUC LR Mate 200iD/7L.

Jerk-bounded Position Controller with Real-Time Task Modification for Interactive Industrial Robots

Jerk-bounded Position Controller with Real-Time Task Modification for Interactive Industrial Robots

Ruixuan Liu, Rui Chen, Yifan Sun, Yu Zhao, Changliu Liu

IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2022

We present JPC, a jerk-bounded position controller that lets industrial robots track complex trajectories while enforcing dynamic constraints at the motion level, without access to servo-level control. JPC also supports real-time task modification: users can replace the ongoing task without violating dynamic constraints.

Iterative Adversarial Data Augmentation

Iterative Adversarial Data Augmentation

Ruixuan Liu, Changliu Liu

ICML Workshop on Human in the Loop Learning, 2021

We propose IADA, an iterative adversarial data augmentation framework for learning neural networks from limited training data. It uses formal verification to find the most “confusing” inputs, then relies on human guidance to safely add them to the training set, one iteration at a time.

Human Motion Prediction using Adaptable Recurrent Neural Networks and Inverse Kinematics

Human Motion Prediction using Adaptable Recurrent Neural Networks and Inverse Kinematics

Ruixuan Liu, Changliu Liu

IEEE Control Systems Letters, 2021

We predict human arm motion during assembly tasks. Online adaptation of a recurrent neural network reduces the prediction error caused by model mismatch and scarce data, while an inverse-kinematics arm model keeps the predicted motion physically feasible.

Automatic Onsite Polishing

Automatic Onsite Polishing

Ruixuan Liu, Weiye Zhao, Suqin He, Changliu Liu

In collaboration with Siemens, Yaskawa Motoman

Weld bead removal is typically done by hand, which is slow, costly, and dangerous in confined spaces such as the inside of a metal tube. We develop a robotic system that polishes and grinds complex surfaces in confined workspaces, reducing cost and manual effort.

Awards

Service

Reviewer for:

Teaching

Other Projects

Course projects, internships, and side projects.

Lidar Obstacle Detection

Lidar Obstacle Detection

Ruixuan Liu

Internship at Zenuity

This was my project as an intern at Zenuity in the summer of 2019. The team was building a mobile robot that could maneuver around the office autonomously. The system included many modules, such as path planning, motion control, and safety. I worked on the lidar perception module. The goal was to use lidar to detect obstacles and compute the free space the robot could traverse.

Basketball Retriever

Basketball Retriever

Ruixuan Liu, Alvin Shi, Roy Li

ECE Capstone: 18-500

This was my undergraduate capstone project. The idea was to build a collaborative mobile robot that helps basketball players during practice. When a player practices shooting, a missed shot can send the ball bouncing anywhere, and retrieving it is frustrating and time-consuming. We built a mobile robot that retrieves the ball each time it bounces away. We would like to thank Prof. Tamal Mukherjee for his guidance and support.

Real-time Object Tracking

Real-time Object Tracking

Ruixuan Liu, Alvin Shi

Computer Vision: 16-720

This was a course project for 16-720: Computer Vision. The goal was to build a pipeline that robustly tracks a target object in real time.

3D Reconstruction with Thermal Texture

3D Reconstruction with Thermal Texture

Ruixuan Liu, Henry Zhang, Yaoyu Hu

Internship at the AirLab

This was a research project I did during my internship in the AirLab led by Prof. Sebastian Scherer. The goal was to enable building inspection from 3D reconstruction. We added thermal texture to the reconstruction to make inspection more reliable.

Methods of Thermal Camera Calibration

Methods of Thermal Camera Calibration

Ruixuan Liu, Henry Zhang, Sebastian Scherer

Internship at the AirLab

This was a research project I did during my internship in the AirLab led by Prof. Sebastian Scherer. We explored techniques to calibrate the intrinsics of a thermal camera.

Line-Following Tank

Line-Following Tank

Ruixuan Liu, Henry Zhang, Alvin Shi

CMU Mobot Competition

★ Third Place

This was a project for the CMU Mobot competition in 2018. This annual competition, held by the School of Computer Science, challenges teams to build a mobile robot that follows a track in front of Wean Hall. Small gates placed along the track serve as checkpoints, and the robot must pass through each of them, so it has to follow the line strictly rather than head directly to the goal.

Gesture Drone

Gesture Drone

Ruixuan Liu, Henry Zhang, Alvin Shi

CMU Build18 Hackathon

This was a project for the 2018 CMU Build18 hackathon. The idea was to use simple, intuitive hand gestures to control a drone. We built a drone from scratch and modified the onboard controller to take custom hand-gesture input.

Smartphone-Controlled Drone

Smartphone-Controlled Drone

Chris Yin, Ruixuan Liu

Internship at HIT

This was a project I did during my internship in the lab led by Prof. Xiaorui Zhu. The goal was to achieve stable, intuitive UAV control based on smartphone motion sensing. This project was done in collaboration with Chris Yin.