ZS.

PhD Candidate · University of Illinois Urbana-Champaign

Zhuchen Shao.

Grounded Interactive Agents

I am a third-year PhD candidate in Electrical and Computer Engineering at the University of Illinois Urbana-Champaign, advised by Prof. Klara Nahrstedt.

My research focuses on grounded, interactive AI agents. I study how agents connect instructions and goals to available evidence, seek missing information through interaction, and use that information to answer questions and take actions. My recent work spans streaming video understanding and video-grounded web agents. I am also exploring grounded collaboration in embodied multi-agent systems.

Zhuchen Shao
Urbana-Champaign, Illinois

Industry experience

Research & systems

Amazon

Alexa AI · Multimodal Agents

May – Aug 2026

Applied Scientist Intern

Developed Tab Zero, a unified browser interface connecting egocentric video understanding with web task execution.

Built and tested a distributed reinforcement learning pipeline with GRPO, SGLang, Megatron-LM, and parallel Chromium rollouts, exploring both rule-based and VLM-based rewards.

Evaluated three model families on Ego2Web and LongEval, including 3–20-minute videos.

Selected research

All publications
VIDEO MEETS THE WEB Tab Zero Web task Act Shared browser actions
Video-grounded web agentsIn preparation

Tab Zero

A unified browser interface that lets web agents inspect egocentric video and complete web tasks using shared browser actions.

Up to 8.0 percentage points higher zero-shot task success rate than uniform sampling on the filtered Ego2Web subset.

STREAMING VIDEO Video stream Hierarchical memory Clarify ↓ Answer
Streaming video understandingWACV, 2027 (accepted)

StreamClarify

A training-free framework for frozen vision-language models, combining entropy-based clarification, temporal-pyramid KV caching, and semantic retrieval.

Built StreamClarify-QA: 1,837 QA pairs from 355 videos. Evaluated three VLM backbones, with macro accuracy gains of up to 8.2 percentage points over existing streaming video QA methods.

WHOLE-SLIDE UNDERSTANDING Slide patches Transformer Context-aware slide classification
Weakly supervised visual learningNeurIPS, 2021

TransMIL

Transformer-based multiple instance learning that models correlations among image patches for whole-slide image classification.

Earlier work on learning from weak supervision, incorporating spatial context into visual representations.

Paper

Full publication list

Papers & manuscripts

11 papers and manuscripts · * Equal contribution

  1. StreamClarify: Proactive Clarification for Grounded Streaming Video Question Answering

    Zhuchen Shao, Bo Chen, Klara Nahrstedt

    WACV, 2027 (accepted)

  2. Tab Zero: Grounding Web Agents in Egocentric Video

    Zhuchen Shao, et al.

    In preparation

  3. Investigating the impact of data consistency in task-informed learned image reconstruction method

    Zhuchen Shao, Changjie Lu, Kaiyan Li, Hua Li, Mark A. Anastasio

    SPIE MI, 2025

    Paper

  4. Prior-guided diffusion model for cell-level segmentation in quantitative phase imaging

    Zhuchen Shao, Mark A. Anastasio, Hua Li

    SPIE MI, 2025

    Paper

  5. Semi-supervised semantic segmentation of cell nuclei with diffusion model and collaborative learning

    Zhuchen Shao, Sourya Sengupta, Mark A. Anastasio, Hua Li

    JMI, 2025

    Paper

  6. AugDiff: Diffusion based Feature Augmentation for Multiple Instance Learning in Whole Slide Image

    Zhuchen Shao, Liuxi Dai, Yifeng Wang, Haoqian Wang, Yongbing Zhang

    TAI, 2024

    Paper

  7. LNPL-MIL: Learning from Noisy Pseudo Labels for Promoting Multiple Instance Learning in Whole Slide Image

    Zhuchen Shao, Yifeng Wang, Yang Chen, Hao Bian, Shaohui Liu, Haoqian Wang, Yongbing Zhang

    ICCV, 2023

    Paper

  8. HVTSurv: Hierarchical Vision Transformer for Patient-level Survival Prediction from Whole Slide Image

    Zhuchen Shao, Yang Chen, Hao Bian, Jian Zhang, Guojun Liu, Yongbing Zhang

    AAAI, 2023

    Paper

  9. dMIL-Transformer: Multiple Instance Learning via Integrating Morphological and Spatial Information for Lymph Node Metastasis Classification

    Yang Chen*, Zhuchen Shao*, Hao Bian*, Zijie Fang, Yifeng Wang, Yuanhao Cai, Haoqian Wang, Guojun Liu, Xi Li, Yongbing Zhang

    JBHI, 2023

    Paper

  10. Multiple Instance Learning with Mixed Supervision in Gleason Grading

    Hao Bian*, Zhuchen Shao*, Yang Chen*, Yifeng Wang, Haoqian Wang, Jian Zhang, Yongbing Zhang

    MICCAI, 2022

    Paper

  11. TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification

    Zhuchen Shao*, Hao Bian*, Yang Chen*, Yifeng Wang, Jian Zhang, Xiangyang Ji, Yongbing Zhang

    NeurIPS, 2021

    Paper

Education

University of Illinois Urbana-Champaign

PhD Candidate, Electrical & Computer Engineering

2024 – 2028 (expected) · Advisor: Klara Nahrstedt

Tsinghua University

M.E., Artificial Intelligence

2021 – 2024 · Advisor: Haoqian Wang

Dalian Maritime University

B.E., Automation

2017 – 2021 · Ranked 1 / 123

Honors

National Scholarship (top 1%)

Tsinghua University · 2023

Dalian Maritime University · 2018, 2019, 2020

Academic service

Reviewer for CVPR, NeurIPS, AAAI, and IEEE TMI.

Recent updates

  • StreamClarify has been accepted to WACV 2027. See you in Orlando!
  • Completed my Applied Scientist internship with Amazon Alexa AI, working on multimodal agents.
  • Passed the PhD qualifying exam and became an ECE PhD Candidate at UIUC.
  • Presented at AAPM 2025 in Washington, D.C.
Earlier updates
  • Gave three presentations at SPIE Medical Imaging in San Diego.
  • Started my PhD in Electrical and Computer Engineering at UIUC.