Portrait of Yunzhe Xu

Yunzhe Xu

Ph.D. Student · Shanghai Jiao Tong University

I study how embodied agents build internal models of their world - remembering what they have seen, imagining what comes next, and coordinating with other agents to act on it. I draw on how the brain solves the same problems, especially the hippocampal link between memory, simulation, and spatial cognition, and use these ideas to build navigation agents that are more capable, efficient, and interpretable. I also work on language models, multimodal LLMs and LLM agents.

I'm a final-year Ph.D. student in the School of Computer Science at SJTU. I lead projects end to end, from choosing the research direction and idea through implementation, experiments, and paper writing, all on a single A100 GPU.

On the job market. I'm looking for postdoctoral positions starting 2027, especially at the intersection of AI and computational neuroscience.

Selected Publications Full list on Scholar →

Memoir overview figure

Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation

Yunzhe Xu, Yiyuan Pan, Zhe Liu

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2026

A world model imagines future states and uses them to query long-term memory of past observations and behaviours, so the agent improves over time without extra supervision. +5.4 SPL over the strongest baseline, with 8.3× faster training and 74% less inference memory.

FLAME: Learning to Navigate with Multimodal LLM in Urban Environments

Yunzhe Xu, Yiyuan Pan, Zhe Liu, Hesheng Wang

AAAI Conference on Artificial Intelligence, 2025 Oral

The first to adapt a multimodal LLM into an outdoor navigation agent, via a three-phase tuning recipe and an LLM-driven synthetic data pipeline. +7.3 points over the previous state of the art, with the entire training done in 14 hours on a single A100 (vs. 672 GPU-hours reported for indoor-VLN work NaVid).

MAVLN overview figure

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

Yunzhe Xu, Zhe Liu

Preprint

The first systematic formalization of multi-agent VLN as constrained coordination; a verified four-stage LLM pipeline yields 11,724 episodes across 145 scenes with teams of up to 4 agents. Our system couples a compact LLM scheduler, a shared topological memory, and conflict-aware execution, and its planner needs only 2.1 GB GPU memory at inference and 21.1 GB for training.

KPPO overview figure

Beyond Elicitation: Provision-based Prompt Optimization for Knowledge-Intensive Tasks

Yunzhe Xu, Zhuosheng Zhang, Zhe Liu

IEEE Transactions on Audio, Speech, and Language Processing (TASLP), 2026

Finds knowledge gaps and injects domain knowledge into LLMs, with adaptive pruning. ~6% average gain across 15 financial, legal, and medical benchmarks with up to 29% fewer inference tokens.

More publications
  • Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigation
    Yiyuan Pan, Yunzhe Xu, Zhe Liu, Hesheng Wang · AAAI 2025 Oral · arXiv
  • Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation
    Yiyuan Pan, Yunzhe Xu, Zhe Liu, Hesheng Wang · NeurIPS 2025 · arXiv
  • Deep Spectral Component Filtering as a Foundation Model for Spectral Analysis Demonstrated in Metabolic Profiling
    Bingsen Xue, Xinyuan Bi, Zheyi Dong, Yunzhe Xu, et al. · Nature Machine Intelligence 2025 · paper

Service

  • Conference reviewerCVPR, ECCV, AAAI, ICRA, CCN
  • Journal reviewerIEEE TASLP, IEEE RA-L, IEEE TMM

Education

  • Ph.D., Computer Science and TechnologyShanghai Jiao Tong University2022 – now
  • B.Eng., Software EngineeringHarbin Institute of Technology · Outstanding Graduate · Graduation thesis 98/100 (1st)2018 – 2022