Education

Education

2022 - Present Brown University, Providence, RI
Ph.D. student in Computer Science
Courses: Computational Linguistics, Introduction to Robotics, Advanced Topics in Deep Learning

2018 - 2021 Northeastern University, Boston, MA
Master of Science in Robotics - Computer Science
Courses: Machine Learning, Reinforcement Learning, Artificial Intelligence, Robotic Science and System, Robotic Sensing and Navigation, Control System Engineering, Mathematical Methods

2014 - 2018 Northeastern University (CN), Shenyang, China
Bachelor of Engineering in Mechanical Engineering
Courses: Control Principles of Mechanical Engineering, Introduction to Robotics technology, C Programming Language, Probability Theory, Machine Design, Sensors & Testing Technology

Research Experience

Jun 2023-Sep 2023 Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Humans to Robots lab, Brown University, Providence, RI

• Proposed a safety constraint module for customizable constraints and integrated the proposed module into an existing LLM agent framework.
• A fully prompting-based approach that supports predicate syntax for translating NL to LTL and explaining violation of LTL specification in NL.
• A formal method-based action pruning and feedback for active re-planning for LLM agents.
• Deployed the whole system in an embodied environment and on real robot platforms and conducted baseline comparisons.

Jan 2023-Jun 2023 Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
Humans to Robots lab, Brown University, Providence, RI

• Proposed a modular system for translating natrual language to linear temporal logic (LTL) for robotic navigation tasks.
• Conducted comprehensive evaluation of the translating tasks by defining different types of holdout tests.
• Deployed translation system and apmdp planner on a Boston Dynamics spot robot.

Jan 2021-May 2022 Improving Instruction Generation for Vision-Language Navigation by Reward Designing
GRAIL lab, Northeastern University, Boston, MA

• Proposed a new method to enhance the performance of language generation model for vision-language navigation by using REINFORCE with multi-modal rewards (BERTScore, VLN agents, ANN scorer)
• Conducted comparison between REINFORCE model with interpolated reward functions and pretrained MLE model with various temperature to approve the effectiveness of policy gradient method on language generation
• Reproduced and improved compatibility model by implementing more robust approach on hard negative mining
• Investigated in training the model with GAN structure by combining the compatibility model with the generation model

Jan-Jul 2021 Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
(Accepted by AI-HRI, 2021) GRAIL lab, Northeastern University, Boston, MA

• Identified shortcomings and opportunities for localization, planning, and language generation in human-robot collaboration in the context of vision-language navigation
• Proposed baseline models in each subtask as reference for further research respectively
• Built dataset for localization task from Room2Room dataset with object detector

Oct 2020-Mar 2021 GASCN: Graph Attention Shape Completion Network
(Accepted by 3DV, 2021) Northeastern University, Boston, MA

• Proposed a novel shape completion model uses graph representation and attention-based permutation-invariant network
• Introduced surface normal and adaptive grid to coarse-to-dense phase for decoding refined point cloud
• Built dataset from ShapeNet by transforming depth images at 8 different camera poses around the object into point cloud
• Carried out performance comparison over different shape completion baselines

Oct 2020 A Study of Policy Gradient Methods for Seq2Seq Model in Neural Machine Translation
Northeastern University, Boston, MA

• Implemented REINFORCE and Actor-Critic in Seq2Seq NMT models training from scratch
• Verified the limitation of using BLEU score as reward in RL methods for NMT both practically and theoretically

Apr 2020 Feature-Based Stereo Visual Odometry
Northeastern University, Boston, MA

• Built odometry pipeline from scratch consisting of feature detection, triangulation, and homogenous transformation
• Processed raw KITTI dataset by undistortion and rectification, and controlled the quality of the result with disparity map

Work Experience

Jan 2021-May 2022 Research Assistant
GRAIL Lab, Northeastern University, Boston, MA

• Enhance the pre-trained speaker model for vision-language generation (VLN) with reinforcement learning
• Rendzevous group project: Combining localization, planning, and language generation

Jan-May 2020 & Jan-May 2021 Teaching Assistant
Northeastern University, Boston, MA

• Configured and tested various robots (e.g., TurtleBot, Duckietown, WidowX) and environments for course project
• Designed coding assignments regarding topics of the course (e.g., robotics kinematics, path planning. robot vision)
• Mentored students on their homework and provided help on their course project accordingly

Jul-Dec 2019 Engineering Tech Co-op
Analogic, Peabody, MA

• Designed and programmed back end module of auto analysis software for implying various system test results on DNA analysis device with python and QT and tested program on Windows Embedded 7
• Conducted and implied modification for better airflow arrangement inside security device for heat dissipation by adding duct system and changing HVAC load
• Designed and implied different tests on DNA analysis device to acquire stronger robustness

Aug-Sep 2017 Assistant Design Engineer
Beiyu MEchanical Equipment Co., Beijing, China

• Learned the principle of different dust extractor units and related installing and commissioning procedures
• Assisted in communicating with Party A and the equipment suppliers
• Calculated tube diameter and pipe resistance