Research Map

Experience

• 

  The University of Queensland

  Incoming PhD student

  School of Electrical Engineering and Computer Science (EECS) · UQMM Lab

  Jul. 2027 – Jan. 2031 Expected
• 

  Shanghai Jiao Tong University

  Research Assistant

  Nov. 2025 – Feb. 2026
• 

  Universiti Malaya

  B.Sc. in Software Engineering

  Oct. 2023 – Mar. 2027 Expected graduation

Preprint / Publications

• 

  arXiv 2026-10

  EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions

  Songruo Chen, Yuqian Fu, Yu Li, Mingkang Dong, Sifan Zhou, Bin Ren, Xu Zheng, Tianwen Qian, Mohamed Elhoseiny, Yanwei Fu

  EgoOmniWorld studies first-person video understanding through interactions between the camera wearer, their actions, and the surrounding world. It organizes evaluation into four structures: Node, Edge, Chain, and Loop, covering individual states, directed relationships, linked dependencies, and feedback cycles. An evidence-grounded pipeline builds 24,166 QA groups from visual and auditory observations, including single-turn questions and multi-turn dialogue. Evaluation exposes uneven capabilities in current multimodal models, while training on the benchmark improves interaction understanding and transfers to other egocentric benchmarks.

• 

  arXiv 2026-08

  ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding

  Mingkang Dong, Muxin Pu, Jie Li, Bohan Guo, Songruo Chen, Bin Ren, Xu Zheng, Chen Zhao, Tianwen Qian, Mohamed Elhoseiny, Yuqian Fu

  Streaming video understanding requires models to retain useful visual evidence before future questions are known. ObjectStream is a training-free framework that treats latent objects as memory anchors: it induces spatially coherent object representations from frozen Video-LLM features, links them across frames into persistent anchors, and maintains bounded histories without external detectors or segmentation models. By preserving persistent object histories, transient object changes, and recent visual context, ObjectStream helps existing Video-LLMs reason over identities, interactions, and state changes more efficiently. Experiments show strong gains on online streaming and offline long-video benchmarks, including improved Qwen2.5-VL-7B performance on OVO-Bench while reducing peak GPU memory and TTFT by about 50%.

• 

  arXiv 2026-06

  Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning

  Mingkang Dong, Hongyi Cai, Xiwen Lei, Jie Li, Tao Zhang, Muxin Pu

  Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection signals from a specific model or dataset, so whenever the target model or candidate pool changes, the criteria must be recomputed from scratch at substantial cost. To address this, we propose OFA, a data selection framework that trains a reusable selector once and applies it to any dataset or model without recomputation. OFA clusters multimodal instructions in a frozen CLIP space, derives pseudo labels from the cluster structure, and trains a lightweight selector for only a few epochs; samples on which this selector is least confident are selected as the most informative. Once trained, the frozen selector transfers directly across datasets and model scales. The selector is trained once on LLaVA-665K and applied both to LLaVA-665K itself and, without any retraining, to the unseen Vision-Flan-186K. Selecting only 15% of the data, OFA achieves 98.3% of full data performance across 10 downstream benchmarks; on the smaller Vision-Flan-186K, the transferred selector surpasses full data training by 10.6%, confirming that the learned signal generalizes to datasets never seen during selector training. The same selected subsets benefit VLMs at both Qwen2.5-VL-3B and LLaVA-v1.5-7B without per model recomputation, decoupling selection from the target model. These results demonstrate that a single, transferable selector provides an effective and reusable solution for efficient multimodal instruction tuning.

• 

  NeurIPS 2026 Proceedings 2026-05

  Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model

  Tao Lin, Yuxin Du, Jiting Liu, Nuobei Zhu, Yunhe Li, Yuqian Fu, Yinxinyu Chen, Hongyi Cai, Zewei Ye, Bing Cheng, Kai Ye, Yiran Mao, Yilei Zhong,Mingkang Dong, Junchi Yan, Gen Li, Bo Zhao

  Vision-Language-Action models have emerged as a promising paradigm for robotic manipulation by unifying perception, language grounding, and action generation. However, they often struggle in scenarios requiring precise spatial understanding, as current VLA models primarily rely on 2D visual representations that lack depth information and detailed spatial relationships. While recent approaches incorporate explicit 3D inputs such as depth maps or point clouds to address this issue, they often increase system complexity, require additional sensors, and remain vulnerable to sensing noise and reconstruction errors. Another line of work explores implicit 3D-aware spatial modeling directly from RGB observations without extra sensors, but it often relies on large geometry foundation models, resulting in higher training and deployment costs. To address these challenges, we propose Evo-Depth, a lightweight depth-enhanced VLA framework that enhances spatially grounded manipulation without relying on additional sensing hardware or compromising deployment efficiency. Evo-Depth employs a lightweight Implicit Depth Encoding Module to extract compact depth features from multi-view RGB images. These features are incorporated into vision-language representations through a Spatial Enhancement Module via depth-aware modulation, enabling efficient spatial-semantic enhancement. A Progressive Alignment Training strategy is further introduced to align the resulting depth-enhanced representations with downstream action learning. With only 0.9B parameters, Evo-Depth achieves superior performance across four simulation benchmarks. In real-world experiments, Evo-Depth attains the highest average success rate while also exhibiting the smallest model size, lowest GPU memory usage, and highest inference frequency among compared methods.

• 

  ECCV 2026 Proceedings 2026-03

  VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning

  Mingkang Dong, Hongyi Cai, Jie Li, Sifan Zhou, Bin Ren, Kunyu Peng, Yuqian Fu

  The effectiveness of multimodal instruction tuning depends not only on dataset scale, but critically on whether training samples genuinely require visual reasoning. However, existing instruction datasets often contain a substantial portion of visually redundant samples (solvable from text alone), as well as multimodally misaligned supervision that can degrade learning. To address this, we propose VisNec (Visual Necessity Score), a principled data selection framework that measures the marginal contribution of visual input during instruction tuning. By comparing predictive loss with and without visual context, VisNec identifies whether a training instance is vision-critical, redundant, or misaligned. To preserve task diversity, we combine VisNec with semantic clustering and select high-necessity samples within each cluster. Across 10 downstream benchmarks, training on only 15% of the LLaVA-665K dataset selected by VisNec achieves 100.2% of full-data performance. On the smaller Vision-Flan-186K dataset, our selection not only further reduces data size but also surpasses full-data training by 15.8%. These results demonstrate that measuring and leveraging visual necessity provides an effective solution for both efficient and robust multimodal instruction tuning. Codes and selected subsets will be released upon acceptance.

• 

  ECCV 2026 Proceedings 2025-11

  Pistachio: Towards Synthetic, Balanced, and Long-Form Video Anomaly Benchmarks

  Jie Li, Hongyi Cai, Mingkang Dong, Muxin Pu, Shan You, Fei Wang, Tao Huang

  Automatically detecting abnormal events in videos is crucial for modern autonomous systems, yet existing Video Anomaly Detection (VAD) benchmarks lack the scene diversity, balanced anomaly coverage, and temporal complexity needed to reliably assess real-world performance. Meanwhile, the community is increasingly moving toward Video Anomaly Understanding (VAU), which requires deeper semantic and causal reasoning but remains difficult to benchmark due to the heavy manual annotation effort it demands. In this paper, we introduce Pistachio, a new VAD/VAU benchmark constructed entirely through a controlled, generation-based pipeline. By leveraging recent advances in video generation models, Pistachio provides precise control over scenes, anomaly types, and temporal narratives, effectively eliminating the biases and limitations of Internet-collected datasets. Our pipeline integrates scene-conditioned anomaly assignment, multi-step storyline generation, and a temporally consistent long-form synthesis strategy that produces coherent 41-second videos with minimal human intervention. Extensive experiments demonstrate the scale, diversity, and complexity of Pistachio, revealing new challenges for existing methods and motivating future research on dynamic and multi-event anomaly understanding.

• 

  CVPR 2026 Highlight 2025-08

  AutoDebias: Automated Framework for Debiasing Text-to-Image Models

  Hongyi Cai, Mohammad Mahdinur Rahman, Mingkang Dong, Jie Li, Muxin Pu, Zhili Fang, Yinan Peng, Hanjun Luo, Yang Liu

  Text-to-Image (T2I) models generate high-quality images from text prompts but often exhibit unintended social biases, such as gender or racial stereotypes, even when these attributes are not mentioned. Existing debiasing methods work well for simple or well-known cases but struggle with subtle or overlapping biases. We propose AutoDebias, a framework that automatically identifies and mitigates harmful biases in T2I models without prior knowledge of specific bias types. Specifically, AutoDebias leverages vision-language models to detect biased visual patterns and constructs fairness guides by generating inclusive alternative prompts that reflect balanced representations. These guides drive a CLIP-guided training process that promotes fairer outputs while preserving the original model's image quality and diversity. Unlike existing methods, AutoDebias effectively addresses both subtle stereotypes and multiple interacting biases. We evaluate the framework on a benchmark covering over 25 bias scenarios, including challenging cases where multiple biases occur simultaneously. AutoDebias detects harmful patterns with 91.6% accuracy and reduces biased outputs from 90% to negligible levels, while preserving the visual fidelity of the original model.

Projects