Yichi Zhang

Ph.D. CandidateUMich CSEConversational AIEmbodied AIMulti-Modal

UMich_Yichi.jpg

Hello! My name is Yichi Zhang, and I am a Ph.D. candidate in Computer Science and Engineering at University of Michigan. I advised by Professor Joyce Chai as a member of the SLED lab. I am broadly interested in the intersection between conversational and embodied AI research, with a particular focus on language grounding to visual and physical contexts, multi-modal dialog, and 3D embodied decision making. I have won the 1st Amazon Alexa Prize SimBot Challenge in 2023 as the team leader of SEAGULL.

Before joining UMich, I obtained my Master’s in Information and Communication Engineering at Tsinghua University in 2020, advised by Professor Zhijian Ou. In 2019, I worked with Professor Zhou Yu as a visiting scholar on task-oriented dialog systems. I got my Bachelor’s in Electronic Information Science and Technology from Tsinghua University in 2017.

News

Jun 17, 2024 Agent-Eval-Refine won the best paper award at CVPR’24 MAR Workshop, and is accepted to COLM 2024!
Feb 26, 2024 GROUNDHOG is accepted to CVPR 2024! see you in Seattle!
Feb 15, 2024 I will join Meta Reality Labs to work with Dr. Shane Moon in Summer 2024 as a Research Scientist Intern. Hope to meet new friends in Seattle!
Oct 07, 2023 Two papers accepted to EMNLP 2023!
Jun 07, 2023 We won the First Place ($500,000) in the 1st Amazon Alexa Prize SimBot Challenge! It was an absolute honor to co-lead the amazing Team SEAGULL with Jed! Big congrats to all of our team members! 🎉 Read our technical report here.

Publications

2024

  1. CVPR
    cvpr24_groundhog.png
    GROUNDHOG: Grounding Large Language Models to Holistic Segmentation
    Yichi Zhang, Ziqiao Ma , Xiaofeng Gao , Suhaila Shakiah , Qiaozi Gao , and Joyce Chai
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Jun 2024
  2. COLM
    colm24.png
    Autonomous evaluation and refinement of digital agents
    Jiayi Pan , Yichi Zhang, Nicholas Tomlin , Yifei Zhou , Sergey Levine , and Alane Suhr
    In First Conference on Language Modeling , Oct 2024

2023

  1. EMNLP
    emnlp23_illusion.png
    Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?
    Yichi Zhang, Jiayi Pan , Yuchen Zhou , Rui Pan , and Joyce Chai
    In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , Dec 2023
  2. EMNLP Findings
    emnlp23_wtag.png
    Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake?
    Yuwei Bao , Keunwoo Yu , Yichi Zhang, Shane Storks , Itamar Bar-Yossef , Alex Iglesia , Megan Su , Xiao Zheng , and Joyce Chai
    In Findings of the Association for Computational Linguistics: EMNLP 2023 , Dec 2023

2022

  1. EMNLP
    emnlp22_danli.png
    DANLI: Deliberative Agent for Following Natural Language Instructions
    Yichi Zhang, Jianing Yang , Jiayi Pan , Shane Storks , Nikhil Devraj , Ziqiao Ma , Keunwoo Yu , Yuwei Bao , and Joyce Chai
    In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Dec 2022

2021

  1. ACL Findings
    acl21_hitut.png
    Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring
    Yichi Zhang, and Joyce Chai
    In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , Aug 2021
  2. EACL
    eacl21_ardm.png
    Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
    Qingyang Wu , Yichi Zhang, Yu Li , and Zhou Yu
    In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , Apr 2021
  3. EMNLP Findings
    emnlp21_trip.png
    Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding
    Shane Storks , Qiaozi Gao , Yichi Zhang, and Joyce Chai
    In Findings of the Association for Computational Linguistics: EMNLP 2021 , Nov 2021
  4. Applied Sciences
    ecrf.png
    Elastic CRFs for Open-Ontology Slot Filling
    Yinpei Dai , Yichi Zhang, Hong Liu , Zhijian Ou , Yi Huang , and Junlan Feng
    Applied Sciences, Nov 2021

2020

  1. AAAI
    aaai20_fig.png
    Task-oriented dialog systems that consider multiple appropriate responses under the same context
    Yichi Zhang, Zhijian Ou , and Zhou Yu
    In Proceedings of the AAAI Conference on Artificial Intelligence , Jan 2020
  2. ACL
    acl20_parg.png
    Paraphrase Augmented Task-Oriented Dialog Generation
    Silin Gao , Yichi Zhang, Zhijian Ou , and Zhou Yu
    In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , Jul 2020
  3. EMNLP
    emnlp20_labes.png
    A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning
    Yichi Zhang, Zhijian Ou , Min Hu , and Junlan Feng
    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Nov 2020
  4. INTERSPEECH
    is2020_dasi.png
    Improved Learning of Word Embeddings with Word Definitions and Semantic Injection.
    Yichi Zhang, Yinpei Dai , Zhijian Ou , Huixin Wang , and Junlan Feng
    In INTERSPEECH , Nov 2020